Multi-touch Interface for Character Motion Control Using Model-Based Approach
Masaki Oshita
Kyushu Institute of Technology
Iizuka, Fukuoka, Japan
Email: oshita@ces.kyutech.ac.jp

Abstract—In this paper, we propose a new method forinteractive motion control with a multi-touch interface. A userof our system can touch and drag character’s body parts tocontrol its motion. The character’s full body motion is driven by
our interactive motion control model based on the movementof a few body parts which are directly manipulated by theuser via the multi-touch interface. We propose a method for
determining 3-dimensional positions of controlled body parts
from 2-dimensional touch inputs based on the character’s
local coordinates and drag speed. We introduce a pointbased pose representation which consists of the positions ororientations of a small number of primary body parts. Basedon the representation, we develop a motion control model that
includes modules for tracking, balance, inter-body interaction,
relaxing and self-collision avoidance. The character’s pose isreconstructed from the point-based pose representation. We
present our experimental results to show that our framework
can realize various natural-looking motions.
Keywords-motion control; multi-touch interface; computer
animation; character animation;

I. I NTRODUCTION
Tablet computers that support multi-touch inputs haverecently become commonplace. Although many applications
control character motion in the virtual environment, mostuse virtual buttons or stroke gestures for selecting actions.
Multi-touch input is a simple substitute for gamepads or
keyboards. The strengths of multi-touch are underutilized.
Using multi-touch, users should be able to control charactermotion freely and intuitively rather than simply executing
predefined actions.
In theory, using inverse kinematics (IK), a user can changea character’s pose by dragging its body part. However, thiskind of interface is not suitable for controlling a character’s
motion in interactive applications, for two primary reasons.
First, because multi-touch inputs on the screen are 2dimensional, 3-dimensional position and orientation of bodyparts cannot be easily controlled. Second, since multiple
body parts must be moved in a coordinated way to realize
natural-looking motion, a user must control multiple bodyparts, which is very difficult. For example, to execute apunch motion, in addition to the hand, the pelvis and trunk
should also be moved.
A statistics-based model for IK (style-based IK [1]) can
be a solution for these problems. Using a large numberof example poses, natural-looking pose and motion can besynthesized based on touch inputs. Oshita [2] applied thisapproach for a multi-touch interface for motion control.
However, because different example data sets are required
for each kind of actions, this approach requires a large
number of examples and a mechanism to switch data sets
automatically. Moreover, it is difficult to execute new typesof actions whose example poses are not provided in advance.
In this paper, we propose a new method for interactive
motion control with a multi-touch interface. A user of oursystem can touch and drag a character’s body parts to control
its motion. Unlike the data-based approach (the statisticsbased IK) discussed above, we take a model-based approach.
The character’s full body motion is driven by our interactive
motion control model based on the movement of a few
body parts which are directly manipulated by the user via
the multi-touch interface. Our motion control model can
perform various motions without preparing any example
poses (Figure 1).
Our method for determining 3-dimensional positions ofcontrolled body parts from 2-dimensional touch inputs isbased on the character’s local coordinates and drag speed.
We introduced a point-based pose representation which
consists of the positions or orientations of a small number of primary body parts (pelvis, hand, foot, trunk andhead). Based on this representation, we developed a motion
control model that includes modules for tracking, balance,inter-body interaction, relaxing and self-collision avoidance.
The character’s pose is reconstructed from the point-basedpose representation. We present our experimental results toshow that our framework can realize various natural-looking
motions.
Even though our method can accept a number of multitouch inputs, in our experience it is difficult for a user tocontrol multiple touches simultaneously. With our method,a user needs to control the movement of only one or a few
primary body parts to perform an action. The motion control
model then generates the full body motion accordingly.
The remainder of this paper is organized as follows.
Section II reviews related work. Section III shows the system
overview. Sections IV, V and VI explain our methods forpose representation, interpretation of multi-touch inputs, andthe motion control model, respectively. Section VII presents
the experimental results and discussion. Finally, Section VIII
concludes this paper.

Figure 1.

Examples of interactive motion control with multi-touch interface. The red circles represent touch inputs.

II. R ELATED W ORK
A. Point-based Pose Representation
We used a point-based pose representation in this research. Representing a pose by using points is an efficient
way to handle character motion. Jakobsen [3] represented
a character as a set of connected particles for an efficientphysics simulation. However, he did not consider reconstructing a full body pose. Popović and Witkin [4] reconstructed full body motion using an optimization process.
These approaches are not applicable to our research.
Similar pose representations to our method which use thepositions and orientations of primary body parts have beenused in previous research [5][6]. The difference between
these studies and our method is that they used their representation for encoding existing motions for motion editing
or retargetting. Kulpa et al. [5] used a numerical IK forlimbs. Neff et al. [6] included the center of mass positionin the pose representation and computed the ankle joints
accordingly during pose reconstruction. We didn’t take suchapproaches to avoid redundancy and achieve robust andefficient method.
B. Motion Control Interface
There are several existing multi-touch interfaces for interactive motion control. Krause et al. [7] applied conventional IK to a character model based on multi-touch inputs
for animation. Kip and Nguyen [8] proposed a system tocontrol one arm and hand of a character using a multitouch interface by changing several parameters to blend armand hand postures. Oshita [2] applied a statistics-based IK(style-based IK [1]). Although this approach successfully
generates natural-looking full-body pose and motion basedon multi-touch inputs, as explained in Section 1, it requiresa large number of examples and a mechanism to switch data
sets automatically. Moreover, it is difficult to execute new
types of actions whose example poses are not provided. On
the other hand, our research used a model-based approachand our motion control model makes it possible to perform
various motions without preparing any example poses.
There are many methods for controlling a character’s
motion by using a single point or trajectory. Generating
locomotion along with a given trajectory is common [9].

However, the type of motion is limited to walking or
running. Throne et al. [10] introduced gesture-based motion selection. Based on gestures drawn along a trajectory,
their system inserts predefined motions such as a jump or
flip. Oshita [11] proposed a stroke-based motion selection
technique that chooses an appropriate action according to
the initial and terminal points of a single stroke drawn on
the screen. With these systems [10][11], users can simply
select actions but the postures and speed of the actions are
fixed and cannot be controlled. Igarashi et al. [12] proposeda spatial keyframing animation technique. This approachenables changing a character’s pose continuously based on
the cursor position. To use this technique, key postures must
be placed at appropriate positions depending on a specificaction.
Some systems allow a user to specify a number of trajectories and constraints for motion creation [13] and deformation[14]. However, with these systems, the user is expected toprioritize the inputs. Our methods uses multiple inputs andcontrol modules and prioritizes them automatically.
C. Motion Control Model
There are various approaches for motion control andsynthesis based on user input. A combination of physics
simulation and physics-based controllers [15][16] is one
approach. A controller determines joint torques based ona target pose, balancing, etc. and the physics simulation
generates physically valid motions. However, designing astable controller is difficult and different controllers must be
designed for each kind of action.
Space-time optimization is another approach [4][17]. It
synthesizes a continuous motion based on given constraints
such as footsteps and timings so that the generated motion
minimizes an objective function that evaluates its physicalvalidity. However, this approach generate a motion sequence
and requires computational time. Therefore, it is difficult toapply it for interactive motion control.
Previous researches has applied optimization (Quadratic
Programming) for computing the pose of the next frame
instead of a motion sequence [18][19] to realize interactive
motion generation. These controllers are designed for autonomous control instead of user control and also requirecomputational time.

Multi-touch inputs

Character Model

Conventional
Pose Representation

Our Point-based
Pose Representation

Interpretation

Constraints

Target Position

Motion Control
Controlled Pose

Pose Reconstruction

Output Pose

Figure 2.

Data flow in the proposed framework.

In contrary to these approaches, we developed a kinematics based controller. Our controller considers physics in partbut directly changes the positions and orientations of bodyparts rather than using physics simulation or optimization.
Our controller is designed to move the character’s full bodynaturally based on the user’s inputs.
III. S YSTEM OVERVIEW
The structure of our framework is shown in Figure 2.
We use an intermediate point-based pose representation formotion control (controlled pose). Multi-touch inputs fromthe user are interpreted and represented as constraints in thesame pose representation.
In addition to constraints for controlling the character’spose, when the character is moved over a certain distance,a moving motion (step) is executed. In this case, the target
position of the moving motion is sent to the motion control
module.
The skeletal structure of the character is given to the system in advance. It includes the information on the shape andweight of the body parts. Shape information is necessary for
self-collision avoidance, and weight information is necessary
for balance control.
The user can also control the camera. A swipe can control
the camera direction and pinch-in and -out can control zoom
(the distance from the camera to the character).
IV. P OINT- BASED P OSE R EPRESENTATION
Our intermediate point-based pose representation P includes pelvis position ppelvis , hand positions pr hand ,pl hand , foot positions pr f oot , pl f oot , trunk orientation
qtrunk and head orientation qhead ; 5 positions and 2 orientations. All positions and orientations are represented in theabsolute (world) coordinates.
In general, there are several ways to represent a rotationsuch as quaternion, axis-angle, 3×3 matrix and Euler angles.
Although any of these can be used with our method, inthe following explanation, we treat them as 3 × 3 matrices
such that the product of two rotations qb qa equates to the
combination of two rotations. q−1 represents the inverse of

Figure 3. Pose representation. A blue rectangle represents a position anda read circle represents a rotation (orientation).

the rotation. In this paper, |q| represents rotational angle andwq represents scaling of the rotational angle.
A. Pose Reconstruction
The character’s output pose X is represented by the
position and orientation of the pelvis and rotations forall joints as X = {ppelvis , qpelvis , qi (i = 1...n)} where
n is the number of joints. Our method reconstructs this
output pose from our intermediate pose representation P ={ppelvis , pr hand , pl hand , pr f oot , pl f oot , qtrunk , qhead }.
Figure 3 shows the difference between the conventionalpose representation and our point-based pose representation.
The pose reconstruction follows 4 steps:
1) Pose Reconstruction for Pelvis Position, Pelvis Orientation and Back Joints: The pelvis position of the intermediate representation is simply used for the pelvis position
ppelvis .
Pelvis orientation and back joint rotations are computedfrom trunk orientation qtrunk . The number of back joints
depends on the skeleton model. In general, the back joint
rotations can be computed from the total rotation of the backjoints by distributing the total rotation to each back joint inspecific ratios [20]. Our method determines pelvis orientation in addition to the back joint rotations. We divide thetrunk orientation into horizontal rotation qtrunk h (1DOF)
and front-back and right-left rotations qtrunk v (2DOF). The
horizontal rotation is assigned to pelvis orientation. The
other rotations are distributed into pelvis orientation andback joints with a specific weight wpelvis back ratio .
qtrunk = qtrunk v qtrunk h(1)

qpelvis = ((wpelvis back ratio )qtrunk v )qtrunk h(2)qi(back) = (1 − wpelvis back ratio )qtrunk v(3)

where wq represents a scaling of rotation angle.
In our implementation, we use wpelvis back ratio = 0.5.
If the skeleton model consists of more than one back joint,
the total back rotation pi(back) can be further distributed toeach back joint with specific weights [20].

selbowq shoulder

pee q eeqwristq elbow

Figure 4.

Analytical inverse kinematics for a limb.

2) Pose Reconstruction for Neck Joints: The neck joint
rotations are computed from trunk and head orientations.
qi(neck) = qhead (qtrunk )−1(4)3) Pose Reconstruction for Limb Joints: After the pelvis
and trunk states are determined, limb joints rotations arecomputed from the end-effector (hand or foot) position. In
the followings explanation, we take the case of an arm to
make the explanation easier, although the same method isapplied to legs too.
In general, as shown in Figure 4, an analytical IK [21]determines the rotation of limb joints (3DOF shoulder joint
qshoulder , 1DOF elbow joint qelbow and 3DOF wrist joint
qwrist ) based on the relative position and orientation of theend-effector from the shoulder pee , qee and swivel angle ofthe elbow selbow . Since we determined the wrist angle joint
rotations differently as explained in the next subsection, if
we focus on shoulder (3DOF) and elbow (1DOF) joints,
their rotations can be determined based on hand positionpee (3DOF) and swivel angle selbow (1DOF).
How swivel angle is determined is important. If analytical
IK is used for changing an existing pose, the swivel angleof the original pose can be retained. Because of the needto generate arbitrary poses in our case, such an approach is
not applicable. It is known that the swivel angle depends onthe end-effector position [22]. Therefore, we determined theswivel angle from the hand position based on examples thatare prepared in advance.
We prepared sets of normalized relative hand positions
and swivel angles as examples. Given a hand position, we
blend nearby samples to determine swivel angle. We use
Radial Basis Function (RBF) to compute the weight of eachexample depending on hand position.
∑
selbow =fi (pee )si(5)fi (p) = exp(−(|pi − pee |/ri )2 ) (if |pi − pee | < ri ) (6)where fi (p) is the RBF for each example {pi , si , ri }. In
our implementation, we use about 10 examples for each limb
which are tuned manually. Alternatively, it is possible to use
motion capture data to create more accurate samples.
Although we used example data here, because theseexamples are common for all types of poses and actions, it is

Figure 5.

Examples of swivel angles.

not necessary to prepare separate sets of examples for eachtype of action, unlike a statistics-based posture synthesis[1][2].
Figure 5 shows poses that are generated from the examples. The colors represent the weights computed from thehand position. Note that each example has a swivel angle
and not a pose.
4) Pose Reconstruction for Hand and Foot Joints: Since
we do not use hand and foot orientations, wrist and ankle rotations pwrist , pankle are determined automatically. Unlessthe character is performing a gesture or holding an object
in the environment, neither is considered in our system, it isnatural to keep the wrist and foot rotations in the rest pose.
Therefore, we simply set the joint rotation to zero after limb
joint rotation is determined.
When an end-effector (typically foot) is contacting the
ground, the foot must be kept horizontal. Also, when the footis near the ground, the ankle joint must be flexed or extended
to prevent the foot from penetrating into the ground. Similar
to trunk orientation control, ankle rotation is set, when thefoot is near or on the ground.
V. I NTERPRETATION OF M ULTI - TOUCH I NPUTSIn this section, we explain how to interpret multi-touch
inputs to determine constraints in the point-based pose
representation form. A user can touch and drag the pelvis,
hand, foot, trunk and head of the character. Since we use
multi-touch inputs, multiple body parts can be touched anddragged at the same time. In this paper, we did not take
inputs on the middle of limbs (e.g. upper arm, forearm,elbow, etc.); limbs can be controlled only by moving the endeffectors (hand and foot). Although a person has 10 fingers,from our experience, we can control at most two or three
touches at the same time. Therefore, the limited numberof controllable body parts in our method is consideredreasonable.
Our method treats each touch input as the spatial translation of the touched body part. The biggest challenge is
how to determine 3-dimensional positions of controlled bodytranslation in the 3D scene
YZ-plane
XY-plane
touch input on the 2D screenytouch linez xtranslation on
the hyper planetranslation in
the perpendicular
direction

Figure 6.

Touch interpretation.

parts. When a user touches and drags a body part on the
2-dimensional screen, the touched position on the screen
represents a line in the 3-dimensional scene. There is no
simple way to determine a unique point on the line.
We solved this problem by introducing several assumptions. First, when a person performs motions, the pelvis,
hand and foot are generally moved in either front-back orright-left direction relative to the person. For example, forpunch and kick motions, the body parts are moved in thefront-back direction. For a waving hand and arm extending
motion, the body parts are moved in the right-left direction.
Although some complex motions include movements incombinations of both front-back and left-right directions, inmany cases the body parts are moved on one plane. We
determined in which plane the touched body part should
be moved depending on the character’s orientation and thecamera direction.
A second assumption is to move the selected body part
(hand or foot) in a perpendicular direction to the screen.
When a hand or foot is moved away from the trunk on
the screen, there is not much freedom of movement in theperpendicular direction, because arm and leg lengths arelimited. However, when the hand or foot is moved towardor near the trunk, it may be moved in the perpendicular direction. When the hand or foot is moved toward the camera,
it is likely that they are dynamic motions such as a punchor kick which would require quick movement. Therefore,
the resultant translation in the perpendicular direction isbased on the speed of the body part being controlled by
the user. Although this may not be the case all of the time,
we consider that this is a reasonable assumption.
In addition to these assumptions, self-collision avoidanceis considered. For example, when a hand is moved toward
the trunk on a plane that crosses the trunk, the arm canpenetrate into the trunk. In this case, the hand positionis adjusted to a position where such penetration does not
happen.

A. Translation of Pelvis, Hand and Foot
Based on the above approach, we determined the target
position of pelvis, hand or foot as follows:
1) Translation on a Hyper Plane: Based on the first
assumption above, when a body part (pelvis, hand or foot)is touched and dragged on the screen, it is moved on either
the XY, ZX or YZ hyper plane which is defined by the local
coordinates of the character as shown in Figure 6.
X, Y and Z axes are determined based on pelvis orientation. Y axis is always (0, 1, 0), while X and Z axes
are determined from the horizontal pelvis orientation. Then,among these axes, the one whose inner product with thecamera vector (camera direction) dcamera is the largest is
chosen as the normal vector that defines the hyper plane. The
hyper plane is defined by the normal vector n and the current
position of the controlled body part pcurrent as followsn(p − pcurrent ) = 0(7)

The touch line where the controlled body part exist is
defined as follows
p = tdtouch + pcamera(8)

where dtouch is the normalized vector from the camera to
the touch point and pcamera is the camera position. Fromequations (7) and (8), the target position of the controlled
body part on the hyper plane is computed.
2) Translation in the Perpendicular Direction: Based on
the second assumption above, position is adjusted in the
direction perpendicular to the screen. This adjustment isapplied to end-effectors (hand and foot) but not to the pelvis.
When the speed of a dragged end-effector is faster than thethreshold, its position is adjusted in proportion to speed.
There are two directions for the perpendicular vector. The
body part should be moved forward from the character. The
moving direction vector df orward is determined as follows.
(if dz axis · dtouch > cos45◦ )(if dz axis · dtouch < cos45◦ )(otherwise)(9)
where dz axis is the Z axis of the character’s pelvis.
Although it is possible to apply this translation on any
camera direction in theory, throughout our tests, we foundthat it does not work well when the body parts are moved
on the YZ plane (when the camera is on the side of thecharacter). Therefore, we limit this control when the angle
between dz axis and dtouch is less than 45◦ .
The position of the end-effector is adjusted in the directionof df orward based on the velocity of the end-effector v.
dtouchdf orward = −dtouch0p′ = p +|v| − vthdf orward (if |v| > vth )
vs

(10)

where vth , vs are the threshold and scaling parameters. We
tuned those parameters empirically. In our implementation,
we use vth = 1.0 and vs = 0.2.
3) Translation for Self-collision Avoidance: Finally, theend-effector is moved in the same perpendicular direction to
avoid self-collision.
The distances between the controlled end-effector andother body parts are computed. If the distance is below a
threshold, the position is adjusted using the similar equation
to equation (10).

pelvistranslation

moving motion(step)

Figure 7.

p′ = p + kdf orward(11)

In this case, the scaling parameter k is computed based on
the distance and the threshold.
For collision detection and distance computation between
body segments, any existing method can be used. The
method for representing the shapes of body segments is
also flexible. A simplified representation such as a boundingbox or ellipsoid makes distance computation easier. In
our implementation, we represented each body part as anoriented bounding box [23].
B. Rotation of Trunk and Head
Because we use the rotations of the trunk and head in
our point-based pose representation, we interpreted the touchand drag of a point on a body part as their rotation.
The translation of the touched point of the trunk or headis computed the same way as the computation for pelvis
translation p′touched . The touched point on the body depends
on the point where the user touches first. Based on thetranslation of the touched point, the trunk rotation thatsatisfies it is computed ∆qtrunk as follows.
(p′touched − pback ) = ∆qtrunk (ptouched − pback )Touch interpretation for the pelvis.

(12)

where ptouched is the initial position of the touched point
without trunk rotation and pback is the back joint position.
Because the touched point is on the surface of the character’sbody, when the touched body is moved downward on thescreen, the character bends forward, and when the touchedbody is moved upward on the screen, it bends backward.
C. Execution of Moving Motion
As explained in Section III, when the pelvis is moved
over a large distance, this is interpreted as moving in a stepand the target position is computed.
Figure 7 shows the touch interpretation for the pelvis.
The horizontal movement of the pelvis is limited within
the support polygon computed from foot positions (seeSection VI-B). Vertical movement of the pelvis is limited
by leg length and minimum duck height. When the pelvis
is moved within this range, the input is interpreted as pelvis
translation. When the pelvis is moved outside this range, theinput is interpreted as moving motion (step) and the target
position is computed.
The target position pmove is computed by computing
the crossing point of the viewing vector with the planethat is parallel to the ground and crosses the pelvis of thecharacter. To keep the step distance within a reasonable
range, the distance between pmove and the current pelvis
position is limited within a specific range (0.5m ∼ 0.8m in
our implementation). Note that up-down touch movement on
the screen controls front-back step direction.
VI. M OTION C ONTROL M ODELIn this section, we describe our motion control model that
determines the output pose Poutput based on the constraints
from multi-touch inputs Pinput and the previous pose Pprev .
These are represented by the intermediate point-based pose
representation described in Section IV.
Our motion control model includes several modules andeach of them determines an output pose. The output pose iscomputed by blending the results of all modules.
Poutput = (∑ wi∑ Pi )
wi(13)

where Pi , wi are the output pose and weight of each control
module and i = tracking, balance, inter body interaction,
relaxing and self collision avoidance. Note that all modules
may not determine all parameters. For example, when there
is no user input, the tracking control does not produce its
output pose and weight. High priority is given to tracking,balance and self collision avoidance while low priority isgiven to inter body interaction and relaxing.
The concepts behind our each control module are not new.
We designed these models based on our intermediate pointbased pose representation. Our models directly change positions and orientation rather than using physics simulation.
We introduced simple approximations and parameters forthese modules. We argue that our framework is very simple
and easy to implement, but also powerful and flexible.
In the remainder of this section, we briefly describe ourapproach for implementing each controller.

A. Tracking Control

D. Relaxing Control

Tracking control is for satisfying input constraints fromthe multi-touch interface. The output of the tracking control
module Ptracking uses input constraints as Ptracking =Pinput .
When an end-effector (hand or foot) is controlled by theuser and the target position is within the reachable range ofthe limb, only the end-effector position is changed. However,when the target position is outside of the reachable range,
the pelvis and the trunk should be moved to reach the target
position. In such case, the pelvis position ppelvis and thetrunk orientation qtrunk are changed accordingly.
Also, while an end-effector (foot) is contacting the
ground, tracking control keeps the current foot position.

Relaxing control is for moving the body parts to therest pose when there is no user input. We introduced thismodule because it looks unnatural if the character stops inan unnatural pose. This module keeps the trunk vertical,
lowers arms and legs, and maintains the head elevation
within certain limits.

B. Balance Control

F. Step Motion Control

Balance control maintains the balance of the character.
Our balance control module controls pelvis position andtrunk orientation to keep the projection of the center of massof the character within its support polygon.
The translation of the upper body (position of the pelvis)∆ppelvis and the rotation of the upper body (orientation of
the trunk) ∆qtrunk are computed to move the center of mass
within the support polygon ∆pcom , when it is found to beoutside the support polygon.

When a step is executed and its target position is sentfrom the interface module, the motion control generates a
step motion. In our implementation, we used a similar modelto that to that used by [24]. During action control, the pelvis
and feet are controlled based on a procedural action model.
The full body motion is automatically generated by our
motion control framework.

M ∆ppelvis = wpelvis ∆pcom
M ∆qtrunk × (ptrunk − pback ) = wtrunk ∆pcom

(14)
(15)

where M is the total mass of upper body. The required
translation of the center of mass ∆pcom is distributed
to the translation of the pelvis position ppelvis and theorientation of the trunk qtrunk with a specific ratio wpelvis: wtrunk = wpelvis trunk ratio : 1 − wpelvis trunk ratio . In
our implementation, we use wpelvis trunk ratio = 0.5.
C. Inter-Body Interaction Control
This control is for simulating the physical interface between connected body parts. When a person moves his or
her body part (e.g. an arm), any connected body part (e.g.
trunk) also moves a little even if he or she tries to keep it still
because there is physical influence between the connectedbody parts.
This module controls trunk orientation based on thevelocities of end-effectors and the positions of end-effectors
based on the velocity of the trunk. Because it is difficult
to simulate this kind of effect accurately even with physics
simulation, because it also requires realistic muscle stiffness
models, we chose to scale the velocities of connected bodyparts.

E. Self-Collision Avoidance Control
Self-collision avoidance requires care because we cannot
know if self-collision will occur until the output pose iscomputed. Therefore, an interim output pose is computed
without self-collision avoidance. If there is any self-collision,
this control is then applied repeatedly until on self-collision
problem exists.

VII. R ESULTS AND D ISCUSSION
We have implemented our interface and tested it. Someof the resulting motions are shown in Figure 1 and theaccompanying video. Various simple actions such as posing,reaching, stepping, gestures (nodding, pointing, waving),
fighting actions (punch, kick) and combinations can becreated by using our interface. As explained in Section I,
even though our method can take multi-touch inputs, it isdifficult for a user to control many parts simultaneously.
Based on our experiments, we found that using one to three
touches at the same time is enough when performing typicalactions. Each action typically has one primary limb (e.g. an
arm for punch). One touch can be used to control its endeffector. Also, one touch can be used to control the body
(pelvis and/or trunk). The rest of the body is driven by our
motion control model.
Using our interface, the user can make his or her avatar
perform their own actions. This will be useful in many
applications such as communication in virtual environments
using avatars or fighting games. Conversely, performing all
motions using our interface may not be realistic. Making
a combination of our interface and conventional interfaces
available and allowing users to choose may be a morepractical option.
There are limitations in our interface. Control is focusedon the pose of a standing character and performing moving
motions other than stepping was not considered in thispaper. There are existing methods for such movement control[9][10] and our system can be integrated with these.

Because of the constraints of using inputs on a 2dimensional screen, it is not possible to control which
direction the character is facing, because the touch inputs areinterpreted as translation or tilt of the controlled body partsand rotation is not considered. It may be possible to extend
our system to include different methods of interpretation or
to make it possible to switch between different interfaces.
However, such extension makes the interface more complicated. Seeking a balance between freedom of control andusability is an important area for future research. Although
our motion control model takes physics into account, it is
not driven by physics simulation and may not always bephysically correct. For example, pose and motion is limited
so that the character never falls. We believe hat this is areasonable constraint, but a user may want the character toperform a falling motion.
VIII. C ONCLUSION
In this paper, we proposed a framework for controllinga character using a multi-touch interface. Although ourframework is very simple, various kinds of motions can berealized using our interface without using any examples. Ourinterface is easy to implement and can be used by many
tablet computers.
R EFERENCES[1] K. Grochow, S. L. Martin, A. Hertzmann, and Z. Popović,“Style-based inverse kinematics,” ACM Transactions on
Graphics, vol. 23, no. 3, pp. 522–531, 2004.
[2] M. Oshita, “Multi-touch interface for character motion control
using example-based posture synthesis,” in International Conference on Computer Graphics, Visualization and Computer
Vision (WSCG) 2012), 2012, pp. 213–222.
[3] T. Jakobsen, “Advanced character physics,” in Proceedings ofGame Developer’s Conference 2001, 2001.
[4] Z. Popović and A. Witkin, “Physically based motion transformation,” in SIGGRAPH 1999, 1999, pp. 11–20.
[5] R. Kulpa, F. Multon, and B. Arnaldi, “Morphologyindependent representation of motions for interactive humanlike animation,” Computer Graphics Forum (Eurographics
2005), vol. 24, no. 3, pp. 343–352, 2005.
[6] M. Neff and Y. Kim, “Interactive editing of motion style using
drives and correlations,” in Eurographics/ACM SIGGRAPH
Symposium on Computer Animation 2009, 2009, pp. 103–
112.
[7] M. Krause, M. Herrlich, L. Schwarten, J. Teichert, andB. Walther-Franks, “Multitouch motion capturing,” in ACM
International Conference on Interactive Tabletops and Surfaces 2008, 2008, p. 2.
[8] M. Kipp and Q. Nguyen, “Multitouch puppetry: Creating
coordinated 3d motion for an articulated arm,” in ACM International Conference on Interactive Tabletops and Surfaces
2010, 2010, pp. 147–156.

[9] S. I. Park, H. J. Shin, and S. Y. Shin, “On-line locomotion
generation based on motion blending,” in ACM SIGGRAPH
Symposium on Computer Animation 2002, 2002, pp. 105–111.
[10] M. Thorne, D. Burke, and M. van de Panne, “Motion doodles:
An interface for sketching character motion,” ACM Transactions of Graphics (SIGGRAPH 2004), vol. 23, no. 3, pp.
424–431, 2004.
[11] M. Oshita, “Motion control with strokes,” Computer Animation and Virtual Worlds, vol. 16, no. 3-4, pp. 237–244, 2005.
[12] T. Igarashi, T. Moscovich, and J. F. Hughes, “Spatial keyframing for performance-driven animation,” in ACM SIGGRAPH/ Eurographics Symposium on Computer Animation 2005,
2005, pp. 253–258.
[13] M. Dontcheva, G. Yngve, and Z. Popović, “Layerd acting forcharacter animation,” in SIGGRAPH 2003, 2003, pp. 409–
416.
[14] B. L. Callennec and R. Boulic, “Interactive motion deformation with prioritized constraints,” Graphical Models, vol. 68,
no. 2, pp. 175–193, 2006.
[15] J. K. Hodgins, W. L. Wooten, D. C. Brogan, and J. F. O’Brien,“Animating human athletes,” in SIGGRAPH ’95, 1995, pp.
71–78.
[16] P. Faloutsos, M. van de Panne, and D. Terzopoulos, “Composable controllers for physics-based character animation,” in
SIGGRAPH 2001, 2001, pp. 251–260.
[17] C. K. Liu and Z. Popović, “Synthesis of complex dynamic
character motion from simple animations,” ACM Transactions
on Graphics (SIGGRAPH 2002), vol. 21, no. 3, pp. 408–416,
2002.
[18] S. Jain, Y. Ye, and C. K. Liu, “Optimization-based interactive
motion synthesis,” ACM Transactions on Graphics, vol. 28,
no. 1, p. Article No. 10, 2009.
[19] A. Macchietto, V. Zordan, and C. R. Shelton, “Momentum
control for balance,” ACM Transactions of Graphics (SIGGRAPH 2009), vol. 28, no. 3, p. Article No. 80, 2009.
[20] G. Monheit and N. I. Badler, “A kinematic model of thehuman spine and torso,” IEEE Computer Graphics and Applications, vol. 11, no. 2, pp. 29–38, 1991.
[21] D. Tolani, A. Goswami, and N. I. Badler, “Real-time inverse
kinematics techniques for anthropomorphic limbs,” GraphicalModels and Image Processing, vol. 62, no. 5, pp. 353–388,
2000.
[22] S. Yonemoto, D. Arita, and R. ichiro Taniguchi, “Real-time
human motion analysis and ik-based human figure control,”in Workshop on Human Motion 2000, 2000, pp. 149–154.
[23] S. Gottschalk, M. Lin, and D. Manocha, “Obbtree: A hierarchical structure for rapid interference detection,” in SIGGRAPH ’96, 1996, pp. 171–180.
[24] C.-C. Wu, J. Medina, and V. B. Zordan, “Simple steps
for simply stepping,” in International Symposium on VisualComputing (ISVC) 2008, 2008, pp. 97–106.