Transactions on Computational Science XXIII, pp. 78-97, Springer, 2014.

Multi-touch Interface and Motion Control
Model for Interactive Character Animation
Masaki Oshita
Kyushu Institute of Technology
Iizuka, Fukuoka, Japan

Abstract. In this paper, we propose a new method for interactive motion control with a multi-touch interface. A user of our system can touchand drag character’s body parts to control its motion. The character’s fullbody motion is driven by our interactive motion control model based on
the movement of a few body parts which are directly manipulated by theuser via the multi-touch interface. We propose a method for determining 3-dimensional positions of controlled body parts from 2-dimensional
touch inputs based on the character’s local coordinates and drag speed.
We introduce a point-based pose representation which consists of the positions or orientations of a small number of primary body parts. Basedon the representation, we develop a motion control model that includes
modules for tracking, balance, inter-body interaction, relaxing and selfcollision avoidance. The character’s pose is reconstructed from the pointbased pose representation. We present our experimental results to showthat our framework can realize various natural-looking motions.
Keywords: motion control, multi-touch interface, computer animation,
character animation

1

Introduction

Tablet computers that support multi-touch inputs have recently become commonplace. Although many applications control character motion in the virtual environment, most use virtual buttons or stroke gestures for selecting actions. Multi-touch input is a simple substitute for gamepads or keyboards. The
strengths of multi-touch are underutilized. Using multi-touch, users should be
able to control character motion freely and intuitively rather than simply executing predeﬁned actions.
In theory, using inverse kinematics (IK), a user can change a character’spose by dragging its body part. However, this kind of interface is not suitablefor controlling a character’s motion in interactive applications, for two primaryreasons. First, because multi-touch inputs on the screen are 2-dimensional, 3dimensional position and orientation of body parts cannot be easily controlled.
Second, since multiple body parts must be moved in a coordinated way to realize
natural-looking motion, a user must control multiple body parts, which is very
diﬃcult. For example, to execute a punch motion, in addition to the hand, thepelvis and trunk should also be moved.

2

Masaki Oshita

Fig. 1. Examples of interactive motion control with multi-touch interface. The redcircles represent touch inputs.

A statistics-based model for IK (style-based IK [1]) can be a solution for these
problems. Using a large number of example poses, natural-looking pose and motion can be synthesized based on touch inputs. Oshita [2] applied this approach
for a multi-touch interface for motion control. However, because diﬀerent example data sets are required for each kind of actions, this approach requires alarge number of examples and a mechanism to switch data sets automatically.
Moreover, it is diﬃcult to execute new types of actions whose example poses are
not provided in advance.
In this paper, we propose a new method for interactive motion control witha multi-touch interface. A user of our system can touch and drag a character’s
body parts to control its motion. Unlike the data-based approach (the statisticsbased IK) discussed above, we take a model-based approach. The character’s
full body motion is driven by our interactive motion control model based on the
movement of a few body parts which are directly manipulated by the user via themulti-touch interface. Our motion control model can perform various motions
without preparing any example poses (Figure 1).
Our method for determining 3-dimensional positions of controlled body parts
from 2-dimensional touch inputs is based on the character’s local coordinates anddrag speed. We introduced a point-based pose representation which consists ofthe positions or orientations of a small number of primary body parts (pelvis,
hand, foot, trunk and head). Based on this representation, we developed a motion
control model that includes modules for tracking, balance, inter-body interaction, relaxing and self-collision avoidance. The character’s pose is reconstructedfrom the point-based pose representation. We present our experimental results
to show that our framework can realize various natural-looking motions.
Even though our method can accept a number of multi-touch inputs, in our
experience it is diﬃcult for a user to control multiple touches simultaneously.
With our method, a user needs to control the movement of only one or a few
primary body parts to perform an action. The motion control model then generates the full body motion accordingly.
This paper is an extended version of our previous work [3]. We describe more
speciﬁc details of our motion control model (Section 6). We also provide a result
from a user test which is a comparison with conventional mouse-based interface
for posing a character (Section 7).

Multi-touch Interface for Interactive Character Animation

3

The remainder of this paper is organized as follows. Section 2 reviews related
work. Section 3 shows the system overview. Sections 4, 5 and 6 explain our
methods for pose representation, interpretation of multi-touch inputs, and the
motion control model, respectively. Section 7 presents the experimental results
and discussion. Finally, Section 8 concludes this paper.

2

Related Work

2.1

Point-based Pose Representation

We used a point-based pose representation in this research. Representing a poseby using points is an eﬃcient way to handle character motion. Jakobsen [4]represented a character as a set of connected particles for an eﬃcient physics
simulation. However, he did not consider reconstructing a full body pose. Popović
and Witkin [5] reconstructed full body motion using an optimization process.
These approaches are not applicable to our research.
Similar pose representations to our method which use the positions and orientations of primary body parts have been used in previous research [6][7]. The
diﬀerence between these studies and our method is that they used their representation for encoding existing motions for motion editing or retargetting. Kulpa
et al. [6] used a numerical IK for limbs. Neﬀ et al. [7] included the center of mass
position in the pose representation and computed the ankle joints accordingly
during pose reconstruction. We didn’t take such approaches to avoid redundancy
and achieve robust and eﬃcient method.
2.2

Motion Control Interface

There are several existing multi-touch interfaces for interactive motion control.
Krause et al. [8] applied conventional IK to a character model based on multitouch inputs for animation. Kip and Nguyen [9] proposed a system to control one
arm and hand of a character using a multi-touch interface by changing several
parameters to blend arm and hand postures. Oshita [2] applied a statistics-based
IK (style-based IK [1]). Although this approach successfully generates naturallooking full-body pose and motion based on multi-touch inputs, as explained in
Section 1, it requires a large number of examples and a mechanism to switch
data sets automatically. Moreover, it is diﬃcult to execute new types of actions
whose example poses are not provided. On the other hand, our research useda model-based approach and our motion control model makes it possible toperform various motions without preparing any example poses.
There are many methods for controlling a character’s motion by using asingle point or trajectory. Generating locomotion along with a given trajectory
is common [10]. However, the type of motion is limited to walking or running.
Throne et al. [11] introduced gesture-based motion selection. Based on gestures
drawn along a trajectory, their system inserts predeﬁned motions such as ajump or ﬂip. Oshita [12] proposed a stroke-based motion selection technique4

Masaki Oshita

that chooses an appropriate action according to the initial and terminal pointsof a single stroke drawn on the screen. With these systems [11][12], users can
simply select actions but the postures and speed of the actions are ﬁxed andcannot be controlled. Igarashi et al. [13] proposed a spatial keyframing animation
technique. This approach enables changing a character’s pose continuously basedon the cursor position. To use this technique, key postures must be placed at
appropriate positions depending on a speciﬁc action.
Some systems allow a user to specify a number of trajectories and constraints
for motion creation [14] and deformation [15]. However, with these systems, theuser is expected to prioritize the inputs. Our methods uses multiple inputs andcontrol modules and prioritizes them automatically.
2.3

Motion Control Model

There are various approaches for motion control and synthesis based on user
input. A combination of physics simulation and physics-based controllers [16][17]is one approach. A controller determines joint torques based on a target pose,balancing, etc. and the physics simulation generates physically valid motions.
However, designing a stable controller is diﬃcult and diﬀerent controllers must
be designed for each kind of action.
Space-time optimization is another approach [5][18]. It synthesizes a continuous motion based on given constraints such as footsteps and timings so thatthe generated motion minimizes an objective function that evaluates its physical validity. However, this approach generate a motion sequence and requires
computational time. Therefore, it is diﬃcult to apply it for interactive motion
control.
Previous researches has applied optimization (Quadratic Programming) forcomputing the pose of the next frame instead of a motion sequence [19][20]to realize interactive motion generation. These controllers are designed for autonomous control instead of user control and also require computational time.
In contrary to these approaches, we developed a kinematics based controller.
Our controller considers physics in part but directly changes the positions andorientations of body parts rather than using physics simulation or optimization.
Our controller is designed to move the character’s full body naturally based on
the user’s inputs.

3

System Overview

The structure of our framework is shown in Figure 2. We use an intermediate
point-based pose representation for motion control (controlled pose). Multi-touch
inputs from the user are interpreted and represented as constraints in the samepose representation.
In addition to constraints for controlling the character’s pose, when the character is moved over a certain distance, a moving motion (step) is executed. In

Multi-touch Interface for Interactive Character Animation
Multi-touch inputs

5Character Model

Interpretation

Constraints

Target Position

Motion Control

Controlled Pose

Pose Reconstruction

Output Pose

Fig. 2. Data ﬂow in the proposed framework.

this case, the target position of the moving motion is sent to the motion control
module.
The skeletal structure of the character is given to the system in advance.
It includes the information on the shape and weight of the body parts. Shapeinformation is necessary for self-collision avoidance, and weight information isnecessary for balance control.
The user can also control the camera using multi-touch interface. A swipe
can control the camera direction and pinch-in and -out can control zoom (thedistance from the camera to the character).

4

Point-based Pose Representation

Our intermediate point-based pose representation P includes pelvis position
ppelvis , hand positions pr hand , pl hand , foot positions pr f oot , pl f oot , trunk
orientation qtrunk and head orientation qhead ; 5 positions and 2 orientations asshown in Figure 3. All positions and orientations are represented in the absolute(world) coordinates.
In general, there are several ways to represent a rotation such as quaternion,axis-angle, 3 × 3 matrix and Euler angles. Although any of these can be usedwith our method, in the following explanation, we treat them as 3 × 3 matrices
such that the product of two rotations qb qa equates to the combination of tworotations. q−1 represents the inverse of the rotation. qv denotes the rotation ofa vector v. In this paper, |q| represents the rotational angle and wq represents
the scaling of the rotational angle.
4.1

Pose Reconstruction

The character’s output pose X is represented by the position and orientation of
the pelvis and rotations for all joints as X = {ppelvis , qpelvis , qi (i = 1...n)} where
n is the number of joints. Our method reconstructs this output pose from our6

Masaki Oshita
conventionalpose representation

our point-basedpose representation

Fig. 3. Pose representation. A blue rectangle represents a position and a read circle
represents a rotation (orientation).

intermediate pose representation P = {ppelvis , pr hand , pl hand , pr f oot , pl f oot ,qtrunk , qhead }. Figure 3 shows the diﬀerence between the conventional pose representation and our point-based pose representation. In our experiments, we useda character model which has 15 joints in the conventional pose representation asshown in Figure 3. The pose reconstruction follows 4 steps as explained below:
Pose Reconstruction for Pelvis Position, Pelvis Orientation and Back
Joints The pelvis position of the intermediate representation is simply used forthe pelvis position ppelvis .
Pelvis orientation and back joint rotations are computed from trunk orientation qtrunk . The number of back joints depends on the skeleton model. In
general, the back joint rotations can be computed from the total rotation of theback joints by distributing the total rotation to each back joint in speciﬁc ratios[21]. Our method determines pelvis orientation in addition to the back joint rotations. We divide the trunk orientation into horizontal rotation qtrunk h (1DOF)
and front-back and right-left rotations qtrunk v (2DOF). The horizontal rotation
is assigned to pelvis orientation. The other rotations are distributed into pelvis
orientation qpelvis and back joints qi(back) with a speciﬁc weight wpelvis back ratio .
In our implementation, we use wpelvis back ratio = 0.5.
qtrunk = qtrunk v qtrunk h(1)

qpelvis = ((wpelvis back ratio )qtrunk v )qtrunk h(2)qi(back) = (1 − wpelvis back ratio )qtrunk v(3)

If the skeleton model contains more than one back joints, the total back rotation qi(back) can be further distributed to each back joint with speciﬁc weights.
The number of back joints of a human character model can be 19 at most [21].
Typically a character model with from one to three back joints is used. In ourMulti-touch Interface for Interactive Character Animation

7

selbow
q shoulder

pee q eeqwristq elbow

Fig. 4. Analytical inverse kinematics for a limb.

implementation, we used a character model with two back joints as depicted in
Figure 3.
Pose Reconstruction for Neck Joints The neck joint rotations are computedfrom trunk and head orientations.
qi(neck) = qhead (qtrunk )−1(4)

Pose Reconstruction for Limb Joints After the pelvis and trunk states are
determined, limb joints rotations are computed from the end-eﬀector (hand orfoot) position. In the followings explanation, we take the case of an arm to makethe explanation easier, although the same method is applied to legs too.
In general, as shown in Figure 4, an analytical IK [22] determines the rotationof limb joints (3 DOF shoulder joint qshoulder , 1 DOF elbow joint qelbow and3 DOF wrist joint qwrist ) based on the relative position and orientation of theend-eﬀector from the shoulder pee , qee and swivel angle of the elbow selbow .
Since we determined the wrist angle joint rotations diﬀerently as explained in
the next subsection, if we focus on shoulder (3 DOF) and elbow (1 DOF) joints,
their rotations can be determined based on hand position pee (3DOF) and swivel
angle selbow (1 DOF).
How swivel angle is determined is important. If analytical IK is used forchanging an existing pose, the swivel angle of the original pose can be retained.
Because of the need to generate arbitrary poses in our case, such an approachis not applicable. It is known that the swivel angle depends on the end-eﬀector
position [23]. Therefore, we determined the swivel angle from the hand position
based on examples that are prepared in advance.
We prepared sets of normalized relative hand positions and swivel angles as
examples. Given a hand position, we blend nearby samples to determine swivel
angle. We use Radial Basis Function (RBF) to compute the weight of eachexample depending on hand position.
selbow =fi (pee )si(5)

8Masaki Oshita

Fig. 5. Examples of swivel angles.

fi (pee ) = exp(−(|pi − pee |/ri )2 ) (if |pi − pee | < ri )(6)

where fi (pee ) is the RBF for each example {pi , si , ri }. In our implementation,
we use about 10 examples for each limb which are tuned manually. Alternatively,
it is possible to use motion capture data to create more accurate samples.
Although we used example data here, because these examples are common
for all types of poses and actions, it is not necessary to prepare separate setsof examples for each type of action, unlike a statistics-based posture synthesis[1][2].
Figure 5 shows poses that are generated from the examples. The colors represent the weights computed from the hand position. Note that each example
has a swivel angle and not a pose.
Pose Reconstruction for Hand and Foot Joints Since we do not use handand foot orientations, wrist and ankle rotations pwrist , pankle are determined
automatically. Unless the character is performing a gesture or holding an object
in the environment, neither is considered in our system, it is natural to keep
the wrist and foot rotations in the rest pose. Therefore, we simply set the joint
rotation to zero after limb joint rotation is determined.
When an end-eﬀector (typically foot) is contacting the ground, the foot must
be kept horizontal. Also, when the foot is near the ground, the ankle joint must
be ﬂexed or extended to prevent the foot from penetrating into the ground.
Similar to trunk orientation control, ankle rotation is set, when the foot is nearor on the ground.

5Interpretation of Multi-touch Inputs

In this section, we explain how to interpret multi-touch inputs to determine
constraints in the point-based pose representation form. A user can touch anddrag the pelvis, hand, foot, trunk and head of the character. Since we use multitouch inputs, multiple body parts can be touched and dragged at the same time.

Multi-touch Interface for Interactive Character Animation

9

In this paper, we did not take inputs on the middle of limbs (e.g. upper arm,
forearm, elbow, etc.); limbs can be controlled only by moving the end-eﬀectors(hand and foot). Although a person has 10 ﬁngers, from our experience, we cancontrol at most two or three touches at the same time. Therefore, the limited
number of controllable body parts in our method is considered reasonable.
Our method treats each touch input as the spatial translation of the touchedbody part. The biggest challenge is how to determine 3-dimensional positionsof controlled body parts. When a user touches and drags a body part on the
2-dimensional screen, the touched position on the screen represents a line in the
3-dimensional scene. There is no simple way to determine a unique point on theline.
We solved this problem by introducing several assumptions. First, when aperson performs motions, the pelvis, hand and foot are generally moved in either front-back or right-left direction relative to the person. For example, forpunch and kick motions, the body parts are moved in the front-back direction.
For a waving hand and arm extending motion, the body parts are moved in theright-left direction. Although some complex motions include movements in combinations of both front-back and left-right directions, in many cases the bodyparts are moved on one plane. We determined in which plane the touched bodypart should be moved depending on the character’s orientation and the camera
direction.
A second assumption is to move the selected body part (hand or foot) in aperpendicular direction to the screen. When a hand or foot is moved away fromthe trunk on the screen, there is not much freedom of movement in the perpendicular direction, because arm and leg lengths are limited. However, when
the hand or foot is moved toward or near the trunk, it may be moved in theperpendicular direction. When the hand or foot is moved toward the camera, it
is likely that they are dynamic motions such as a punch or kick which would require quick movement. Therefore, the resultant translation in the perpendicular
direction is based on the speed of the body part being controlled by the user.
Although this may not be the case all of the time, we consider that this is areasonable assumption.
In addition to these assumptions, self-collision avoidance is considered. For
example, when a hand is moved toward the trunk on a plane that crosses thetrunk, the arm can penetrate into the trunk. In this case, the hand position isadjusted to a position where such penetration does not happen.
5.1

Translation of Pelvis, Hand and Foot

Based on the above approach, we determined the target position of pelvis, handor foot as follows:
Translation on a Hyper Plane Based on the ﬁrst assumption above, whena body part (pelvis, hand or foot) is touched and dragged on the screen, it is
moved on either the XY, ZX or YZ hyper plane which is deﬁned by the local
coordinates of the character as shown in Figure 6.

10

Masaki Oshitatranslation in the 3D scene

touch input on the 2D screenyYZ-plane
XY-plane
touch linez xtranslation on
the hyper planetranslation in
the perpendicular
direction

Fig. 6. Touch interpretation.

X, Y and Z axes are determined based on pelvis orientation. Y axis is always(0, 1, 0), while X and Z axes are determined from the horizontal pelvis orientation. Then, among these axes, the one whose inner product with the camera
vector (camera direction) dcamera is the largest is chosen as the normal vector
that deﬁnes the hyper plane. The hyper plane is deﬁned by the normal vector n
and the current position of the controlled body part pcurrent as followsn(p − pcurrent ) = 0(7)

The touch line where the controlled body part exist is deﬁned as follows
p = tdtouch + pcamera(8)

where dtouch is the normalized vector from the camera to the touch point andpcamera is the camera position. From equations (7) and (8), the target positionof the controlled body part on the hyper plane is computed.
Translation in the Perpendicular Direction Based on the second assumption above, position is adjusted in the direction perpendicular to the screen.
This adjustment is applied to end-eﬀectors (hand and foot) but not to the pelvis.
When the speed of a dragged end-eﬀector is faster than the threshold, its positionis adjusted in proportion to speed.
There are two directions for the perpendicular vector. The body part should
be moved forward from the character. The moving direction vector df orward isdetermined as follows.
dtouch (if dz axis · dtouch > cos45◦ )df orward = −dtouch (if dz axis · dtouch < cos45◦ )0(otherwise)(9)

Multi-touch Interface for Interactive Character Animation

11

where dz axis is the Z axis of the character’s pelvis. Although it is possible toapply this translation on any camera direction in theory, throughout our tests,
we found that it does not work well when the body parts are moved on the YZ
plane (when the camera is on the side of the character). Therefore, we limit thiscontrol when the angle between dz axis and dtouch is less than 45◦ .
The position of the end-eﬀector is adjusted in the direction of df orward basedon the velocity of the end-eﬀector v.
|v| − vthdf orward (if |v| > vth )(10)vswhere vth , vs are the threshold and scaling parameters. We tuned those parameters empirically. In our implementation, we use vth = 1.0 and vs = 0.2.
p = p +Translation for Self-collision Avoidance Finally, the end-eﬀector is moved
in the same perpendicular direction to avoid self-collision.
The distances between the controlled end-eﬀector and other body parts arecomputed. If the distance is below a threshold, the position is adjusted using thesimilar equation to equation (10).
p = p + kdf orward(11)

In this case, the scaling parameter k is computed based on the distance and thethreshold.
For collision detection and distance computation between body segments,
any existing method can be used. The method for representing the shapes ofbody segments is also ﬂexible. A simpliﬁed representation such as a boundingbox or ellipsoid makes distance computation easier. In our implementation, we
represented each body part as an oriented bounding box [24].
5.2

Rotation of Trunk and Head

Because we use the rotations of the trunk and head in our point-based pose
representation, we interpreted the touch and drag of a point on a body part as
their rotation.
The translation of the touched point of the trunk or head is computed thesame way as the computation for pelvis translation ptouched . The touched pointon the body depends on the point where the user touches ﬁrst. Based on thetranslation of the touched point, the trunk rotation that satisﬁes it is computed
Δqtrunk as follows.
(ptouched − pback ) = Δqtrunk (ptouched − pback )(12)

where ptouched is the initial position of the touched point without trunk rotationand pback is the back joint position. Because the touched point is on the surface ofthe character’s body, when the touched body is moved downward on the screen,
the character bends forward, and when the touched body is moved upward on
the screen, it bends backward.

12

5.3

Masaki Oshita

Execution of Moving Motion

pelvistranslation

moving motion(step)

Fig. 7. Touch interpretation for the pelvis.

As explained in Section 3, when the pelvis is moved over a large distance,
this is interpreted as moving in a step and the target position is computed.
Figure 7 shows the touch interpretation for the pelvis. The horizontal movement of the pelvis is limited within the support polygon computed from footpositions (see Section 6.2). Vertical movement of the pelvis is limited by leg
length and minimum duck height. When the pelvis is moved within this range,
the input is interpreted as pelvis translation. When the pelvis is moved outside this range, the input is interpreted as moving motion (step) and the target
position is computed.
The target position pmove is computed by computing the crossing point of
the viewing vector with the plane that is parallel to the ground and crosses thepelvis of the character. To keep the step distance within a reasonable range,
the distance between pmove and the current pelvis position is limited within a
speciﬁc range (0.5m ∼ 0.8m in our implementation). Note that up-down touch
movement on the screen controls front-back step direction.

6

Motion Control ModelIn this section, we describe our motion control model that determines the output
pose Poutput based on the constraints from multi-touch inputs Pinput and theprevious pose Pprev . These are represented by the intermediate point-based pose
representation described in Section 4.
Our motion control model includes several modules and each of them determines an output pose. The output pose is computed by blending the results ofall modules.
 wi
 Pi )
Poutput = ((13)
wiMulti-touch Interface for Interactive Character Animation

13

where Pi , wi are the output pose and weight of each control module and i =tracking, balance, inter body interaction, relaxing and self collision avoidance.
Note that all modules may not determine all parameters. For example, when
there is no user input, the tracking control does not produce its output pose andweight. High priority is given to tracking, balance and self collision avoidance
while low priority is given to inter body interaction and relaxing.
The concepts behind our each control module are not new. We designed thesemodels based on our intermediate point-based pose representation. Our models
directly change positions and orientation rather than using physics simulation.
We introduced simple approximations and parameters for these modules. We
argue that our framework is very simple and easy to implement, but also powerful
and ﬂexible.
In the remainder of this section, we brieﬂy describe our approach for implementing each controller.
6.1

Tracking Control

Tracking control is for satisfying input constraints from the multi-touch interface.
The output of the tracking control module Ptracking uses input constraints as
Ptracking = Pinput .
When an end-eﬀector (hand or foot) is controlled by the user and the target
position is within the reachable range of the limb, only the end-eﬀector position
is changed. However, when the target position is outside of the reachable range,
the pelvis and the trunk should be moved to reach the target position. In suchcase, the pelvis position ppelvis and the trunk orientation qtrunk are changed
accordingly as follows.
Δproot = pinput
root − proot(Δqtrunk (ptrunk − pback ) − (ptrunk − pback )) + Δppelvis = Δproot

(14)
(15)

where Δproot is the required translation of the root of the limb (shoulder).
ptrunk and pback are the positions of the trunk and back joints, respectively.
First, the trunk orientation Δqtrunk is computed to satisfy Δproot as muchas possible. Then, the pelvis translation Δppelvis is computed to satisfy theremaining required translation Δproot .
Also, while an end-eﬀector (foot) is contacting the ground, tracking control
keeps the current foot position. When no constraint is speciﬁed to a body part,the tracking control does not handle the body part, as result, the body part iscontrolled by other modules.
Since the priority of tracking control should be high, wtracking is set to 1.0(the highest weight).
6.2

Balance Control

Balance control maintains the balance of the character. Our balance control
module controls pelvis position and trunk orientation to keep the projection of
the center of mass of the character within its support polygon.

14

Masaki Oshita

First, based on the character’s current state, the position of the center ofmass and its projection to the ground pcom is computed. The support polygon
S is also computed based on the positions and contact conditions of the feet.
Then it is determined if pcom is within S. If not, the closest point to pcom within
S is computed as pcom . To maintain the character’s balance, the center of mass
is moved from pcom to pcom .
Δpcom = pcom − pcom

(16)

The translation of the upper body (position of the pelvis) Δppelvis and therotation of the upper body (orientation of the trunk) Δqtrunk are computed tomove the center of mass within the support polygon Δpcom , when it is found to
be outside the support polygon.
M Δppelvis = wpelvis Δpcom
M (Δqtrunk (ptrunk − pback ) − (ptrunk − pback )) = wtrunk Δpcom

(17)
(18)

where M is the total mass of upper body. The required translation of thecenter of mass Δpcom is distributed to the translation of the pelvis position
ppelvis and the orientation of the trunk qtrunk with a speciﬁc ratio wpelvis :wtrunk = wpelvis trunk ratio : 1 − wpelvis trunk ratio . In our implementation, we
use wpelvis trunk ratio = 0.5.
Since the priority of balance control is as high as tracking control, wbalanceis set to 1.0 too.
6.3

Inter-Body Interaction Control

This control is for simulating the physical interface between connected bodyparts. When a person moves his or her body part (e.g. an arm), any connectedbody part (e.g. trunk) also moves a little even if he or she tries to keep it still
because there is physical inﬂuence between the connected body parts.
This module controls trunk orientation based on the velocities of end-eﬀectors
and the positions of end-eﬀectors based on the velocity of the trunk. Because it
is diﬃcult to simulate this kind of eﬀect accurately even with physics simulation,
because it also requires realistic muscle stiﬀness models, we chose to scale thevelocities of connected body parts.
The change of trunk orientation Δqtrunk is computed as follows.
vi = (pi − pprev
)/Δt
i

(19)

Δqtrunk (ptrunk − pback ) − (ptrunk − pback ) = wee to trunk Δtvi

(20)

where pi , pprev
are the current and previous end-eﬀector positions, wee to trunkiis the inﬂuence from the end-eﬀector velocity to the trunk orientation. In our
implementation, we use wee to trunk = 2.0.
The end-eﬀector position is computed as follows.

Multi-touch Interface for Interactive Character Animation15

vi = ((Δqtrunk (pi − ptrunk ) − (pi − ptrunk )) + (ppelvis − pprev
pelvis ))/Δt(21)pi = pi + wtrunk to ee Δtvi

(22)

We use wtrunk to ee = 0.2. This is not applied to the end-eﬀector (feet) that iscontacting on the ground.
Since the priority of this control is low, a low value is set to winteraction (0.1
in our implementation).
6.4

Relaxing Control

Relaxing control is for moving the body parts to the rest pose when there is nouser input. We introduced this module because it looks unnatural if the character
stops in an unnatural pose. This module keeps the trunk vertical, lowers arms
and legs, and maintains the head elevation within certain limits.
The trunk orientation is computed by
qtrunk = qtrunkv qtrunkh
r = qtrunkv −1
qtrunkhqtrunk = vtrunk Δt|r|(if |r| ≤ vtrunk Δt)
rqtrunk (if otherwise)

(23)
(24)
(25)

where qtrunkh and qtrunkv are decomposition of the trunk rotation into thehorizontal orientation and the other component. The rotation r makes the trunk
orientation closer to qtrunkh in a predeﬁned angular speed vtrunk .
Each end-eﬀector positions is computed bypi = pi + Δtg (if pi,y > hee )(26)

where g = (0, vee , 0) is a predeﬁned spatial speed. This equation is applied when
the position of the end-eﬀector is higher than the predeﬁned height hee .
When there is no input to the head. The head is moved based on the trunk.
However, this does not look natural. When a person moves, the head elevation
is kept horizontal. To realize such head movements, the head orientation qhead
is computed by the same equation with equation (25).
Since the priority of this control is low, a low value is set to wrelaxing (0.1 inour implementation).
There are more sophisticated methods for gazing (head and eye orientation)
control. However, such control is not considered in our motion control model,
because as such methods work only on a virtual environment with other characters and objects which can be gazing targets and those are not considered inour current system.

16

6.5

Masaki Oshita

Self-Collision Avoidance Control

Self-collision avoidance requires care because we cannot know if self-collision
will occur until the output pose is computed. Therefore, an interim output poseis computed without self-collision avoidance. If there is any self-collision, thiscontrol is then applied repeatedly until on self-collision problem exists.
This module solves self-collision between limbs and trunk by moving limbs.
Assuming the limbs didn’t penetrate into the trunk on the previous frame, theself-collision can be ﬁxed by moving the limb (end-eﬀector position) toward thestate of the previous frame.
pi = tpi + (1 − t)pprev
i(27)

where t(0 < t < 1). t is reduced from 1 to 0, until the collision is resolved.
However, in case that the trunk is also moving, moving the limbs to theprevious state may not solve the collision. In such case, the limb (end-eﬀector
position) is moved outward as follows.
pi = pi + (pi − proot) × dzi(28)where proot
is the position of the root of the limb (shoulder) and dz is the zi

axis of its local coordinates.  is a small displacement for each step. This step isrepeated until the self-collision is avoided.
Note there is the diﬀerence between this module and the method describedin Section 5.1. The former deals with self-collision during motion control, while
the latter is for interpreting multi-touch inputs. Both are important for not only
avoiding self-collision but also for generating natural motion from the touch anddrag inputs.
Since the priority of this control is high, wself collision avoidance is set to 1.0.

6.6

Step Motion Control

When a step is executed and its target position is sent from the interface module,
the motion control generates a step motion. In our implementation, we used asimilar model to that to that used by [25]. During action control, the pelvis andfeet are controlled based on a procedural action model. The full body motion is
automatically generated by our motion control framework.
The step control determines trajectories of the pelvis and feet. First, the goal
goal
position of both feet pgoalr f oot , pl f oot are computed from the given goal positionof the pelvis pgoal
pelvis so that the relative positions is kept before and after step
as follows.
goal
initialinitialpgoal
r f oot = ppelvis + (pr f oot − ppelvis )(29)

Our step control generate one cycle of step, that is, a leg is moved ﬁrst andthe other leg follows. The leg that is near to the target position is moved ﬁrst.
For example, when the character steps to the right, the right leg is moved ﬁrst.
The ﬁrst moving leg is determined based on the target position.

Multi-touch Interface for Interactive Character Animation

17

Fig. 8. Mouse-based interface for comparison. The position and orientation of a joint
or end-eﬀector can be controlled by dragging a handle.

The step action is executed in two phases; preparation and execution phases.
During preparation phase, the pelvis is moved over the ﬁrst supporting leg sothat the character can move the moving leg. During execution phase, one cycleof step is executed. The trajectories of the pelvis and feet during those phases
can be approximated by linear trajectories [25]. The durations of those phases
are determined based on the length of the trajectories.

7

Results and Discussion

We have implemented our interface and tested it. Some of the resulting motions
are shown in Figure 1 and the accompanying video. Various simple actions suchas posing, reaching, stepping, gestures (nodding, pointing, waving), ﬁghting actions (punch, kick) and combinations can be created by using our interface. As
explained in Section 1, even though our method can take multi-touch inputs, it
is diﬃcult for a user to control many parts simultaneously. Based on our experiments, we found that using one to three touches at the same time is enoughwhen performing typical actions. Each action typically has one primary limb
(e.g. an arm for punch). One touch can be used to control its end-eﬀector. Also,
one touch can be used to control the body (pelvis and/or trunk). The rest of the
body is driven by our motion control model.
We conducted a user test to show the eﬃciency of our multi-touch interface.
Because there is no alternative common interface that can control full-body
motion of a charter interactively, it is diﬃcult to compare our interface for motion
control tasks. Therefore, in this user study we evaluated our interface for posing
control tasks. We compared our interface with a conventional mouse-based pose
editing interface where a joint rotation or position can be manipulated using
handles displayed on the screen as shown in Figure 8. The blue and red sphereson the character represent controllable end-eﬀectors and joints, respectively. Thiskind of interface is commonly used in commercial animation systems such as
maya, max and softimage. Although only one body can be controlled at a time,
its position and rotation can be controlled precisely.

18

Masaki Oshita

1

2

3

4

5

Fig. 9. Target poses used in the user test.
Table 1. Results from the user test. Average times for making each target pose.
Pose no. Mouse interface Multi-touch interface
1
27.4 s8.6 s2
33.2 s9.9 s
334.7 s11.9 s4
76.2 s25.6 s5132.5 s40.1 sSeven subjects who have basic knowledge of computer animation participatedin the user test. During the experiment, a target pose was showed to the subject
and he or she was asked to make the same pose by using one of the interfaces.
The system determines that the task is completed when the maximum distance
between the positions of all joints in the target and controlled poses becomes
below the threshold (10 cm in our experiment). The ﬁve target poses used in our
experiment are shown in Figure 9. The required times for making each target
pose from the initial pose were measured. Each subject used both interfaces.
The average times for the posing tasks are shown in Table 1. For all target
poses, our multi-touch interface was more eﬃcient than the mouse-based interface. From our observation, the subjects ﬁrst tried to make the lower body poseby using one or two touches and by changing the view direction if necessary.
They then made the upper body pose by using multi-touch. This is because therelaxing control module lowers the arms if the user stops touching them. Sincebody orientation cannot be controlled by our multi-touch interface, it seemedthat the subjects found it diﬃcult to make some poses such as pose no. 4. The
subjects also found it diﬃcult to control end-eﬀector positions when two hands
are close to each other such as pose no. 4, because multiple touches cannot point
the same position. The pose no. 5 took time the most, because the all feet andhands must be controlled both in front-back and right-left directions.
Using our interface, the user can make his or her avatar perform their own
actions. This will be useful in many applications such as communication in virtual environments using avatars or ﬁghting games. Conversely, performing allMulti-touch Interface for Interactive Character Animation

19

motions using our interface may not be realistic. Making a combination of ourinterface and conventional interfaces available and allowing users to choose may
be a more practical option.
There are limitations in our interface. Control is focused on the pose of astanding character and performing moving motions other than stepping was notconsidered in this paper. There are existing methods for such movement control[10][11] and our system can be integrated with these.
Because of the constraints of using inputs on a 2-dimensional screen, it is
not possible to control which direction the character is facing, because the touch
inputs are interpreted as translation or tilt of the controlled body parts androtation is not considered. It may be possible to extend our system to include
diﬀerent methods of interpretation or to make it possible to switch between different interfaces. However, such extension makes the interface more complicated.
Seeking a balance between freedom of control and usability is an important areafor future research. Although our motion control model takes physics into account, it is not driven by physics simulation and may not always be physicallycorrect. For example, pose and motion is limited so that the character neverfalls. We believe hat this is a reasonable constraint, but a user may want thecharacter to perform a falling motion.

8

Conclusion

In this paper, we proposed a framework for controlling a character using a multitouch interface. Although our framework is very simple, various kinds of motions
can be realized using our interface without using any examples. Our interface iseasy to implement and can be used by many tablet computers.

Acknowledgment
This work was supported in part by a Grant-in-Aid for Scientiﬁc Research (No.
24500238) from the Japan Society for the Promotion of Science (JSPS).

References
1. Grochow, K., Martin, S.L., Hertzmann, A., Popović, Z.: Style-based inverse kinematics. ACM Transactions on Graphics 23(3) (2004) 522–531
2. Oshita, M.: Multi-touch interface for character motion control using example-based
posture synthesis. In: International Conference on Computer Graphics, Visualization and Computer Vision (WSCG) 2012. (2012) 213–222
3. Oshita, M.: Multi-touch interface for character motion control using model-based
approach. In: International Conference on Cyberworlds 2013. (2013) 330–337
4. Jakobsen, T.: Advanced character physics. In: Proceedings of Game Developer’s
Conference 2001. (2001)
5. Popović, Z., Witkin, A.: Physically based motion transformation. In: SIGGRAPH
1999. (1999) 11–20

20

Masaki Oshita

6. Kulpa, R., Multon, F., Arnaldi, B.: Morphology-independent representation of
motions for interactive human-like animation. Computer Graphics Forum (Eurographics 2005) 24(3) (2005) 343–352
7. Neﬀ, M., Kim, Y.: Interactive editing of motion style using drives and correlations.
In: Eurographics/ACM SIGGRAPH Symposium on Computer Animation 2009.
(2009) 103–112
8. Krause, M., Herrlich, M., Schwarten, L., Teichert, J., Walther-Franks, B.: Multitouch motion capturing. In: ACM International Conference on Interactive Tabletops and Surfaces 2008. (2008) 2
9. Kipp, M., Nguyen, Q.: Multitouch puppetry: Creating coordinated 3d motion for
an articulated arm. In: ACM International Conference on Interactive Tabletops
and Surfaces 2010. (2010) 147–156
10. Park, S.I., Shin, H.J., Shin, S.Y.: On-line locomotion generation based on motion
blending. In: ACM SIGGRAPH Symposium on Computer Animation 2002. (2002)
105–111
11. Thorne, M., Burke, D., van de Panne, M.: Motion doodles: An interface for sketching character motion. ACM Transactions of Graphics (SIGGRAPH 2004) 23(3)(2004) 424–431
12. Oshita, M.: Motion control with strokes. Computer Animation and Virtual Worlds
16(3-4) (2005) 237–244
13. Igarashi, T., Moscovich, T., Hughes, J.F.: Spatial keyframing for performancedriven animation. In: ACM SIGGRAPH / Eurographics Symposium on Computer
Animation 2005. (2005) 253–258
14. Dontcheva, M., Yngve, G., Popović, Z.: Layerd acting for character animation. In:
SIGGRAPH 2003. (2003) 409–416
15. Callennec, B.L., Boulic, R.: Interactive motion deformation with prioritized constraints. Graphical Models 68(2) (2006) 175–193
16. Hodgins, J.K., Wooten, W.L., Brogan, D.C., O’Brien, J.F.: Animating humanathletes. In: SIGGRAPH ’95. (1995) 71–78
17. Faloutsos, P., van de Panne, M., Terzopoulos, D.: Composable controllers forphysics-based character animation. In: SIGGRAPH 2001. (2001) 251–260
18. Liu, C.K., Popović, Z.: Synthesis of complex dynamic character motion from simple
animations. ACM Transactions on Graphics (SIGGRAPH 2002) 21(3) (2002) 408–
416
19. Jain, S., Ye, Y., Liu, C.K.: Optimization-based interactive motion synthesis. ACM
Transactions on Graphics 28(1) (2009) Article No. 10
20. Macchietto, A., Zordan, V., Shelton, C.R.: Momentum control for balance. ACM
Transactions of Graphics (SIGGRAPH 2009) 28(3) (2009) Article No. 80
21. Monheit, G., Badler, N.I.: A kinematic model of the human spine and torso. IEEE
Computer Graphics and Applications 11(2) (1991) 29–38
22. Tolani, D., Goswami, A., Badler, N.I.: Real-time inverse kinematics techniques
for anthropomorphic limbs. Graphical Models and Image Processing 62(5) (2000)
353–388
23. Yonemoto, S., Arita, D., ichiro Taniguchi, R.: Real-time human motion analysis
and ik-based human ﬁgure control. In: Workshop on Human Motion 2000. (2000)
149–154
24. Gottschalk, S., Lin, M., Manocha, D.: Obbtree: A hierarchical structure for rapid
interference detection. In: SIGGRAPH ’96. (1996) 171–180
25. Wu, C.C., Medina, J., Zordan, V.B.: Simple steps for simply stepping. In: International Symposium on Visual Computing (ISVC) 2008. (2008) 97–106