Multi-Touch Interface for Character Motion Control
Using Example-Based Posture Synthesis
Masaki Oshita
Kyushu Institute of Technology
680-4 Kawazu, Iizuka, Fukuoka
820-8502, Japanoshita@ces.kyutech.ac.jp

Figure 1. Example of the multi-touch interface for character motion control.

ABSTRACT
We propose a multi-touch interface for character motion control with which a user can control a character’s posefreely and make the character perform various actions by simply dragging the character’s body parts using a
multi-touch input device. We use style-based inverse kinematics to synthesize a natural-looking pose thatsatisfies given constraints using a learned model of sample postures. However, the style-based inverse
kinematics is suited to posture editing and not motion control. It cannot handle various types of actions andcannot generate continuous and physically valid motions. To overcome these limitations, we prepare different
learned models for each action and choose an appropriate learned model according to the user’s input. Morespecifically, we use a single learned model for posing and different learned models for each type of action. We
also use different motion generation methods for posing and action controls. Furthermore, we introduce tracking
control as a post-process after posing and action controls to generate continuous and physically plausible motion.
We implemented the proposed method using a Windows 7 Touch API on a multi-touch-enabled personal
computer, and demonstrated the power of our interface.

Keywords
Motion control, multi-touch interface, example-based posture synthesis, physics-based control.

1. INTRODUCTION
The flexibility of character motion control ininteractive applications is currently very limited.
Since common input devices such as a gamepad,
mouse or keyboard have only a small number ofdegrees of freedom, the user can do nothing but
simply select an action from a few pre-defined
actions by pressing an associated button and make acharacter perform a fixed action such as walking,Permission to make digital or hard copies of all or part ofthis work for personal or classroom use is granted without
fee provided that copies are not made or distributed forprofit or commercial advantage and that copies bear thisnotice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or toredistribute to lists, requires prior specific permission
and/or a fee.

punching, or kicking. It is impossible for users topose the character freely or make the character
perform various actions in the user’s own style. Thisis particularly a limitation in some interactive
applications such as fighting games, dance animationand online communication using avatars. For
example, users may want a character to perform
various moves in fighting games and interactive
dance animation or use various gestures incommunication in a multi-user network environment.
We propose a multi-touch interface for charactermotion control with which a user can control acharacter’s pose freely and make the character
perform various actions by simply dragging thecharacter’s body parts using a multi-touch input
device (Figure 1).
In theory, using inverse kinematics (IK), a user could
control a character’s pose in detail. However, with

conventional input devices, the user can control only
one end effector at a time. Moreover, it is difficult torealize natural-looking motion using IK even with amulti-touch device because multiple body parts must
be controlled in a coordinated way to execute an
action.
Our method uses style-based IK [GMH*04][SL06].
Style-based IK constructs a learned model of poses
in advance by mapping a large number of sample
poses onto a low-dimensional latent space. It thensynthesizes a natural-looking pose that satisfies given
constraints using the learned model. However, stylebased IK has several problems. First, it cannot handle
many types of actions with a single learned model
because appropriate poses depend on the type ofaction to be expressed. Second, it is difficult togenerate continuous motion from given touch strokesof body parts because there is no guarantee that acontinuous trajectory in Cartesian space is mapped toa continuous trajectory in latent space. Third,
resulting motions that are synthesized using stylebased IK lack physical validity, since the charactercan take any pose at any speed without considering
physical constraints. For these reasons, style-based
IK cannot be simply used for the motion control ofvarious types of actions. Basically, style-based IK issuitable for making a pose but not for interactive
motion control.
To solve these problems, we prepare different
learned models for each action and choose anappropriate learned model according to the user’s
input. More specifically, we use a single learned
model for posing and different learned models foreach kind of action. We also use different motion
generation methods for posing and action controls.
During posing control, a pose is generated simply
using a learned model based on multi-touch inputs.
When a touch stroke for a body part matches theinitial trajectory of the primary body part of aprepared action model, the system switches to action
control. During action control, we control time
progression by considering conditions of sample
postures with a learned model to generate continuousand natural-looking motion. We also introduce
tracking control as a post-process for posing andaction controls to generate continuous and physicallyplausible motions. Using physics simulation,
physical effects such as the shaking of noncontrolled body parts by controlled body parts arecalculated and applied.
We implemented the proposed method using
Windows 7 Touch API on a multi-touch-enabled
personal computer, and thus demonstrated the powerof our interface.
The rest of this paper is organized as follows. In
Section 2, we review related works. Section 3

presents an overview of our method, while Sections
4, 5 and 6 describe posing, action and tracing
controls respectively. Results and a discussion arepresented in Section 7. Finally, Section 8 concludes
the paper.

2. RELATED WORK
Multi-touch Interface for Motion Control
To our knowledge, there are only a few researchworks that use a multi-touch interface for motion
control and generation. Krause et al. [KHS*08]
applied conventional IK to a character model basedon multi-touch inputs for animation. However, as
explained in the previous section, it is difficult torealize complex motions with this approach.
Moreover, animation takes a long time andinteractive control is impossible. Kip and Nguyen[KN10] proposed a system to control one arm andhand of a character using a multi-touch interface bychanging several parameters to blend arm and handpostures. However, their system is limited to thecontrol of one arm and one hand and cannot be used
to control full-body motion.

Trajectory-Based Motion Generation
Since the mouse and pen were common input devices
well before multi-touch devices became available,
there has been much research on the use of a single
point or trajectory to control or generate charactermotion.
A common way to use trajectory for motion control
is to use a trajectory to specify a locomotion path
[PSS02]. Many animation systems have a function to
generate walking and running motions according to a
trajectory drawn on the ground. However, with thistype of interface, the type of motion is limited towalking or running motion.
Throne et al. [TBvdP04] introduced gesture-based
motion selection. Based on the gestures drawn along
a trajectory, their system inserts predefined motions
such as a jump or flip. Oshita [Osh05] proposed astroke-based motion selection technique that chooses
an appropriate action according to the initial andterminal points of a single stroke drawn on the screen.
With these two systems, users can simply selectactions by drawing a trajectory or stroke, but
postures and the speed of actions are fixed andcannot be controlled.
Igarashi et al. [IMH05] proposed a spatial
keyframing animation technique. By placing key
poses in the three-dimensional space in advance, a
new posture is synthesized by blending the key
postures according to the distances between thecurrent mouse position and the key postures. By
drawing a trajectory on the screen (moving themouse cursor), a continuous motion can be generated.

However, to generate natural-looking and intended
motion, the key postures must be placed at
appropriate positions and at appropriate distances
and the user must move the mouse cursor properly.
This requires training and careful design. It is alsodifficult to generate various types of action with asingle set of key postures when using this method.
Dontcheva et al. [DYP03] used a physical threedimensional marker to specify body trajectories.
However, since the user can control one body part ata time with their system, they need to repeat
specifying trajectory for each body part. Theirmethod is for off-line animation making and is notsuitable for interactive control.

Style-based IK
As explained in Section 1, style-based IK generates a
posture according to user input. The approach is notsuitable for motion generation or control. Previous
application of style-based IK is limited to posture
editing [GMH*04] and motion generation based on
trajectory in the latent space [SL06]. Motion
generation based on trajectory in three-dimensional
space or on a screen was not realized.
Min et al., [MCC09] introduced a statistics-basedmodel of motion instead of postures. Using a given
trajectory of a specified body part, sample motions
are blended to synthesize new motion. The method
can be used with multi-touch devices. However,since entire motions are blended, detailed control
such as time and speed control and changing postures
is not possible. Moreover, the model must be learned
for a specific type of action and the system does notallow the execution of combinations of various typesof actions.

Other motion control interfaces
Various devices have been used for interactive
motion control by researchers. Some physical sensors
such as Wii remote (acceleration sensors) can beused with gesture recognition technique [LLZ*09][BNT08]. By performing a specific gesture, the user
can select an action from predefined actions.
However, this type of gesture-based interface simply
substitutes for conventional interfaces such as gamepad and keyboard. Control of the character’s fullbody motion and executing action with user’s own
style are not possible.
Some researches combine physics simulation with acontrol of limited number of degrees of freedom.
Laszlo et al. [LZS05] used mouse to control a few
joints and generated full-body motion by using
physics simulation. Shiratori and Hodgins [SH08]
used Wii remotes to determine a few parameters forphysics-based controllers of several actions such as
walking, running, and jumping. By using physics
simulation, physically plausible motion can be

Multi-touch input
Constraints
body part and position for each touch

Learned models（action control）

Learned model(posing control)
Posing control

Action control

Target posture
Tracking control
Output posture

Figure 2. System overview.
generated and controlled. However, the physicsbased controllers must be designed for each type ofaction in advance and realizing various types ofactions and styles is difficult.
Recently, low-price markerless motion capturedevices such as Microsoft Kinect have become
available. Using such a device, the character’s fullbody motion can be freely controlled [IWZ*09][Osh06]. However, such a device requires a larger
workspace. Moreover, to perform highly dynamic
actions, the user must actually perform the actions.
This is difficult for non-trained users.

3. SYSTEM OVERVIEW
The overview of our system is shown in Figure 2.
The system generates a character’s posture in eachframe according to multi-touch inputs. Any type ofmulti-touch device can be used with our system.
When the user touches a body part of the characterand drags it on the screen, each touch input ishandled as a constraint to control the character. Notethat each constraint is a two-dimensional position on
the screen; that is, a half-line in the threedimensional space.
Our system uses posing and action control modes
with different learned models. Under the initialcondition, the posing control mode is used to change
the character’s posture according to multi-touch
inputs. When the trajectory of the body part that isdragged by the user matches the trajectory of apredefined action, the system switches to the action
control mode to generate an action that dynamicallychanges according to the multi-touch inputs. Whenthe action finishes, the system switches back to theposing control mode.
In addition to posing and action controls, tracking
control is applied as a post-process to change the(a) Posing control(b) Action control

Figure 3. Example of latent space (twodimensional).

posture generated by posing and action controls sothat the generated motion is continuous andphysically plausible.
Since our system handles only the touching anddragging of character’s body parts, other types ofmulti-touch inputs such as touching and dragging inanother area, and multi-touch gestures including
pinching and swiping, can be used for otherfunctions such as viewpoint control depending on theapplication.

4. POSING CONTROLThe posing control is used to generate charactermotions that have no particular form such as achange in the standing pose or the free movement ofhands and feet. For posing control, we use a method
similar to conventional style-based IK. The problemsof style-based IK are solved by combining action andtracking controls. In our implementation, we use amethod similar to that used by Shin et al. [SL06].
As sample postures for the learned model in posing
control, we use postures extracted from motion datafor various poses. In addition, the initial period of
motion data for action control is also used, sinceinitial parts of actions are realized by posing control
before action control is initiated.

Learned model for posing control
To apply style-based IK, sample postures are mappedon a low- (typically two- or three-) dimensional
latent space to construct a learned model in advance.
There are various ways to map example postures
onto a latent space [SL06]. In our implementation,
we employed multi-dimensional scaling (MDS). The
sample postures are mapped to a latent space so thatthe distances between sample postures are preservedin the latent space. The distance between any two
postures is calculated according to the sum of
distances between all pairs of corresponding joints
after two postures are aligned according to theirpelvis position and orientation in the same manner asin [KGP02].

Figure 4. Example of trajectories of action
models.

Posture synthesis during posing control
From the given constraints, which are twodimensional positions of body parts, a posture issynthesized using the learned model by blending
sample postures in the latent space.
We divide the latent space into a grid and assign a
representative sample posture to each grid in advanceas shown in Figure 3, where each dot represents asample posture. By comparing the representative
sample postures with given constraints, thecorresponding cell can be found efficiently. Since theconstraints consist of a two-dimensional half line of
selected body parts, the distance between given
constraints and a sample posture can be calculated
from the average of the distance between the half lineof the constraints and the position of thecorresponding body part in the sample posture. Oncethe corresponding cell is determined, the distance
between each sample posture in the cell and given
constraints is calculated to determine the blending
weight of each sample posture. By blending thesample postures with the weights, a posture issynthesized. For posture blending, rotation of eachjoin in the sample postures is blended using
quaternions [PSS02]. For pelvis position andorientation, the relative position and orientation takenfrom the initial state in original motion data are
blended. The blended position and orientation areadded to the character’s original position andorientation when the posing control begins.
The synthesized posture is generated by blending
sample postures so that the posture satisfies the given
constraints as much as possible. However, since thenumber of sample postures is limited compared with
the number of possible posture configurations, there
is no guarantee that the constraints are satisfied.
Therefore, we further apply numerical IK to thesynthesized posture. However, if body parts are
moved a large distance, the modified posture may
become unnatural. Therefore, we limit the distance towithin a certain distance (0.2 m). In addition, the footpositions of the synthesized posture may not matchthe current posture. We also apply conventional IK
to fix the foot position when the foot is on the ground.
In our implementation, we use cyclic coordinate
descent (CCD) IK [Wel93].

5. ACTION CONTROL
Action control is used to generate a character’s
motion with a particular form such as punching,kicking, or jumping. We prepare a specific learned
model for each action. An action model is selected
depending on user inputs. During action control, we
control time progression by considering conditions ofsample postures to generate continuous and naturallooking motions.

Executing condition for action control
The executing condition for an action model isdetermined according to the trajectory of a body parton the screen when the user drags the body part.
Each action model has a primary body part (e.g., theright hand for a right-hand punching action) and its
average trajectory, which is calculated from the
motion data used to train the action model. The
primary body part is manually specified. The
trajectory of the primary body part taken from eachaction model is projected onto the screen andcompared with the input trajectory.
When the primary body part is dragged by the user,
the distance between the user input trajectory and the
trajectory of each action model is calculated todetermine whether the action is to be initiated. A
trajectory is represented by a series of pairs of thetwo-dimensional screen point and time. To calculate
the distance between two trajectories, corresponding
points from two trajectories are first determinedusing a dynamic programming algorithm. The
average distance is then calculated from thedifference in positions and difference in relative
times for each pair of corresponding points:
D   t input,i  t action,j  h  tinput,i  taction,j , (1) i , j S i , j S

where t input,i , tinput,i , t action,j taction,j are the positions andtimes of the points for the input and action
trajectories, S is the set of corresponding points for
two trajectories, and h is a scaling parameter. Whenthe computed distance D is lower than a threshold,the action is initiated. This process is applied only
when the input trajectory has certain length andS covers more than half the action trajectory.
In our preliminary user test, we found that users tend
to drag a body part quickly when they want toexecute an action while they drag body parts slowly
for pose control. Therefore, we change the thresholdallowed tostillautomatic
direction

allowedbackward

not allowed
forwardFigure 5. Conditions during an action.
depending on the speed of the touch input. When
body parts are moved quickly, an action is initiatedeven if the distance between the trajectories is large.
The three-dimensional trajectory of each action
model is projected onto a screen. Therefore,
determining action execution is view-dependent.
Depending on the current viewpoint, the trajectory
may be normal to the screen and the projectedtrajectory may be too short. This makes the executionof action difficult and introduces recognition errors.
For example, it is difficult to draw the trajectory of
the hand for a punching action when the character isfacing the viewpoint. To solve this problem, we
compute a three-dimensional vector that represents
the trajectory of the action model in advance andcalculate the angle between the representative vector
and viewpoint vector (eye vector). If the angle issmaller than a threshold (45 degrees in our
implementation), the action is excluded from theaction execution process. For example, when the user
wants a character to perform a punching action, theuser must control the viewpoint so that the user can
see the character from the side or top before the user
drags the character’s hand, because a punchingaction cannot be executed when the view is toward
the front or back of the character. We believe that
this is a reasonable constraint because it is impossible
to draw a trajectory of punching action from the front
or back anyway, and it is natural for the user tochange viewpoint so that the user can draw a
trajectory properly. Figure 4 shows the trajectories ofaction models. The red trajectories represent
executable actions while the pink trajectories
represent non-executable actions from the current
viewpoint.

Conditions on sample postures
We also use style-based IK as a fundamental
technique for action control. During action control,
the generated motion must follow the particular formof action. For example, when the user drags thecharacter’s right hand forward quickly during apunching action, the character should not suddenly
move the hand to the specified position but shouldperform the punching motion until the right handreaches the specified position. Moreover, action is
not simply executed forward; it can also be stopped
or executed backward depending on the user’s
control.
However, a stopping action should not be allowed
when it results in a physically impossible motion. For
example, a real human cannot stop still during a kickor jump. Even when the user stops controlling (i.e.,
all fingers are removed from the screen), a propermotion should be generated. The action should beeither cancelled by being executed backward or
finished by being executed forward automatically.
To realize such control, each sample posture has two
independent conditions: a stillness condition that indicates whether thecharacter is allowed to become still when thetouch input stops moving but stays on the screen,and an automatic execution direction that indicates
whether motion is to be executed forward or
backward when the touch input stops and leaves
from the screen.
To set these conditions, we specify the time segmentsof conditions on each motion as shown in Figure 5.
For all motions associated with an action, the sametypes and order of segments are specified. Eachsample posture is expressed in generalized time (0 to1). In Figure 3(b), the color of the dot represents thegeneralized time of the sample posture (blue to red).
Although it may be possible to determine these
conditions automatically, we currently chose to set
this information manually, since it does not takemuch time.
The learned model for each action is constructed in
the same way as the model for posing control.
Postures derived from motion data for the action
model are used. In our implementation, we use a few(one to four) motions for each action model. An
example of latent space for an action (kicking) isshown in Figure 3(b), where series of postures (dots)
are connected and the color of the dots represents thegeneralized time (red to blue).

Posture synthesis during action control
During action control, the continuity of generated
postures is considered. At first, in the same way as
for posing control, the latent space coordinates are
determined from given inputs (constraints on bodyparts). The constraints can include those on the
primary body part and those on other body parts.
Using a multi-touch device, positions or trajectories
of several body parts can be specified. At the sametime, the generalized time is computed from thesample postures near the coordinates and weights. If

the generalized time for the synthesized posture isclose to the current time, the synthesized posture isused as output. Otherwise, the current time isforwarded or rewound toward the synthesized time ina certain time interval. The interval is determinedaccording to the execution frame rate (normally
about 1/30 seconds). The same weights for sample
postures taken from sample motions are used togenerate a synthesized posture with the updated time.
If the state at the current generalized time does notallow stillness, the time keeps being forwarded or
rewound, even if the user’s finger is not moving on
the screen. If the user’s finger is lifted from thescreen, the time is forwarded or rewound dependingon the state at the current time. During suchautomatic forwarding or rewinding, the last weights
for sample motions are kept and used for posturesynthesis.
When the action control reaches the end of motion,
the system switches back to the posing control. The
discontinuity from switching control is fixed using
tracking control.

6. TRACKING CONTROL
Tracking control is used as a post-process after
posing or action control to generate continuous andphysicallyplausible
motion.
We
employacceleration-based tracking to generate continuous
motion and physics-based tracking to add a physicaleffect to the generated motion.
Physics-based tracking control has been widely usedin previous research [ACS*07][ZMC*05][HP97].
Proportional derivative (PD) control is a simple andpopular method that calculates an output torque foreach joint according to the current and target statesof the character. A physics simulation then updates
the character’s state according to the calculated
torques. However, this approach has several
problems. For PD controllers to work, appropriate
gain parameters must be given for each joint.
Appropriate parameters vary depending on motion
and posture. Even though there are several methods
for determining gain parameters semi-automatically[ACS*07][ZMC*05][HP97], it remains difficult torealize stable control, especially for highly dynamic
motions such as jumping and kicking. Anotherproblem is that the motions generated with thisapproach lag the target motion and are too slow andsmooth. It is difficult to realize a target motion
correctly while keeping the details of the target
motion. To solve these problems, we only extract
physical effects such as non-controlled body parts
being shaken by controlled body parts from the resultof physics-based tracking control. We use an
acceleration-based method [Osh06] for moreaccurate and faster tracking control. By combining

the two tracking controls, continuous and physicallyplausible motions are generated.

Acceleration-based tracking
Acceleration-based tracking uses PD controllers to
calculate rotational acceleration according to thecharacter’s current state and the target posturecalculated from posing or action control. The angularacceleration for each joint is calculated as
i  k  q target, i  q i   d q i ,q(2)

i , q i , qi are the joint rotational acceleration,
where qvelocity and rotation, respectively, q target, i is thetarget joint rotation, and k and d are gain anddamping parameters. We use a quaternion torepresent joint rotations. The minus symbol denotes
the calculated difference between two rotations. The
scaling operation scales the rotation of quaternion.
Since we use rotational accelerations instead of joint
torques, we can use the same k and d for all joints.
In addition to joint rotations, the spatial accelerationof the root segment (pelvis) is calculated: root  k  p target, root  p root   dp root ,p(3)

 root , p root , p root are the pelvis acceleration,
where pvelocity and position, respectively.

From the calculated acceleration, the character’s joint
rotations and rotational velocity are updated:
i  q i , qi  t q i  q i ,q i  t q

(4)

where q i , qi are the values for the previous frame.

q i  s1q i  s2 qsim,i  q sim,i ,(6)where q i is the relative rotation for the physicaleffect, qsim,i is the joint rotation calculated in thephysics simulation, qi , qsim,i are the values for theprevious frame, and s1 , s2 are attenuation parameters(we use s1  0.2t , s2  0.5t ). Physical effects suchas vibration caused by the movement of other joints
are calculated using equation (6).
We apply this physical effect to the body parts thatare not controlled by the user and are not in contact
with the ground. We divide the character’s body into
five segments: two legs, two arms and the torso. For
each segment, the above condition is evaluated todetermine whether the physical effects are applied as:
q output,i  qi  q i ,(7)where q output,i is the output joint rotation, qi is thejoint rotation calculated by the acceleration-based
tracking control (equation (4)), and q i is therelative joint rotation calculated by the physics-based
tracking control (equation (6)). The same process isapplied to the pelvis position and rotation. Afterward,
for the legs in contact with the ground, we apply IK
to fix the foot positions according to the changed
pelvis position and orientation.

7. RESULTS AND DISCUSSION

Combining physics-based tracking
In parallel with acceleration-based tracking control,
physics-based tracking control is carried out.
Physics-based tracking uses PD controllers andphysics simulation at first. In our implementation, we
use Open Dynamics Engine (ODE) for physics
simulation. For tracking control, we apply forces toeach body segment according to the difference
between current and target positions:fi  k  p target,i  pi   dp i ,from physics-based control, we extract velocities andapply the accumulation of the velocities withattenuation as physical effects. The physical effects
are calculated as(5)

where pi , p i are the position and velocity of the i-th
segment, p target,i is the target position, and fi is theforce applied to the segment. The positions andvelocities are updated according to the forces
calculated in physics simulation. Other physicalproperties such as gravity and contact forces between
the feet and ground are considered during physics
simulation. These forces are automatically handled
by the simulation engine.
As explained above, we extract physical effects fromthe results. Instead of using the posture generated

We implemented the proposed method using a
Windows 7 Touch API on a multi-touch-enabled PC.
We are also developing an iOS (iPad/iPhone/iPodtouch) version. Currently, our system has about 10
action models such as models for punching, kicking,bowing, waving a hand, jumping, and walking. The
accompanying video shows various motion controls
using our interface.
In a preliminarily user test, we asked a professional
animator to trial our system to determine if oursystem can be used for animation. He understood theconcept of our system and was able to use the system
immediately, including the requirement of changing
the viewpoint depending on the action. He noted thelimitations mentioned below and that the generated
posture was sometimes unnatural. This is basically
due to a lack of postures that we used for learningmodels, and the system can be improved by adding
more motion data. The professional animator agreed
that a multi-touch interface is useful and noted
especially that he was able to fix body parts bycontinuing to touch them. He commented that itmight be difficult to create final motions with oursystem in an interactive way because the creation
requires editing that is more precise, but our system
is a promising tool with which to generate initialmotions to be edited later with other animation
systems.

executed in our current framework. However, torealize accurate control of continuous walking
directions and paths, an additional method is required.
There are animation systems that generate walking
motions according to a given trajectory [PSS02], andthey can be integrated with our system.

In the trial, we found out that a single touch is almost
enough to control various actions. In general, as
several body parts must be moved in a coordinated
way to realize an action, multi-touch control isconsidered to be necessary. However, because weintroduced novel methods to realize actions such astechniques for the switching of actions and action
control, various actions can be easily realized bymoving a single body part. Nevertheless, multi-touch
control is useful when the user wishes to add changes
to postures during action by moving secondary bodyparts. It is also necessary to change posturespecifically during posing control by imposing
constraints on multiple body parts. The user
sometimes wishes to fix some body parts duringposing or action control, and our multi-touch
interface can be used not only to move body parts but
also to fix them. The beauty of our method is that it
integrates single-touch and multi-touch interfaces.
Many types of motion generation are possible with asingle touch while detailed control is possible withmultiple touching. Users can thus take advantage ofboth types of touch.

Since there is no alternative system that allows
various controls as our interface does, it is difficult tomake comparisons with other methods. However,conducting a user test to determine whether novice
and professional users are able to use our interface
easily is future work. Application of our methods tosoftware that uses full-body motion control of acharacter and testing the effectiveness of our method
are also future work.

Similarly to other pen-based or touch-screen-based
interfaces [IMH05][MCC09][Osh05][PSS02][TBvdP04], our interface has several problems relating to the
input device. First, when touching the screen, thescreen is occluded by the user’s hands. Anotherproblem is that the user cannot give a command forthe next action while the character is still executing
the previous action, since it is difficult to touch anddrag a moving body part. These are fundamental
problems that are difficult to solve. However, theproblems can be reduced by introducing additional
interfaces and methods. For example, the use ofmultiple screens can solve the occlusion problem. By
displaying a virtual future posture on the screen andallowing input to the posture, the problem of input
delay can be solved.
A limitation of our method is that users can execute
only predefined actions. By adding more motion data
for new actions, our system can handle other types ofaction. However, it is difficult to combine two
actions such as waving a hand while walking. To
realize this type of combination, an additional action
model for each combination must be given inadvance. Blending of multiple action models during
action control is future work. Another issue is thecontrol of walking and running. As shown in theaccompanying video, a cycle of walking can be

8. CONCLUSION
We proposed a multi-touch interface for charactermotion control. Unlike conventional interfaces withwhich the user can only select an action from precreated actions, our interface allows free posturecontrol and the performing of various predefined
actions with the user’s own styles. Our methods canbe used in interactive applications such as computer
games and online communication. Using ourinterface, users will be able to express themselves bymoving in their own style. In addition, our methods
can be used for animation. Recently, multi-touch
devices such as smart phones, tablet computers, andLCD monitors with multi-touch sensors have become
popular. We believe that our methods will be a
powerful interface for applications on such multitouch devices.

ACKNOWLEDGMENTS
This work was supported in part by Research Grant
from Hayao Nakayama Foundation as well as
Grants-in-Aid for Scientific Research (24500238)
from the Japan Society for the Promotion of Science(JSPS).

REFERENCES[ACS*07] Brian Allen, Derek Chu, Ari Shapiro andPetros Faloutsos. On the Beat! Timing andTension for Dynamic Characters. ACM
SIGGRAPH/Eurographics
SymposiumonComputer Animation 2007, pp.239-247, 2007.
[BNT08] Ronan Billon, Alexis Nedelec, Jacques
Tisseau. Gesture Recognition In Flow based on
PCA Analysis using Multiagent System. ACM
SIGCHI International Conference on Advances in
Computer Entertainment Technology 2008 (ACE2008), pp. 139-146, 2008.
[DYP03] Mira Dontcheva, Gray Yngve, Zoran
Popovic: Layered Acting for Character
Animation. ACM Transactions of Graphics

(SIGGRAPH 2003), Vol. 22, Issue 3, pp. 409416, 2003.
[GMH*04] Keith Grochow, Steven L. Martin, Aaron
Hertzmann, Zoran Popović. Style-based Inverse
Kine-matics. ACM Transactions on Graphics,
Vol. 23, Issue 3, pp. 522-531, 2004.
[HP97] Jessica K. Hodgins, and Nancy S. Pollard.
Adapting Simulated Behaviors For New
Characters. SIGGRAPH 1997, pp.153-162, 1997.
[IMH05] Takeo Igarashi, Tomer Moscovich, John F.
Hughes. Spatial Keyframing for Performancedriven Animation. ACM SIGGRAPH /Eurographics
SymposiumonComputer
Animation 2005, pp. 253-258, 2005.
[IWZ*09] Satoru Ishigaki, Timothy White, Victor
Zordan, C. Karen Liu. Performance-BasedControl Interface for Character Animation. ACM
Transactions of Graphics (SIGGRAPH 2009),
28(3), Article No. 61, 2009.
[KGP02] Kucas Kovar, Michael Gleicher, Fedderic
Pighin. Motion Graphs. ACM Transactions on
Graphics (SIGGRAPH 2002), Vol. 21, Issue 3,
pp. 473-482, 2002
[KN10] Michael Kipp, Quan Nguyen. Multitouch
Puppetry: Creating coordinated 3D motion for anarticulated arm. ACM International Conferenceon Interactive Tabletops and Surfaces 2010, pp.
147-156, 2010.
[KHS*08] Markus Krause, Marc Herrlich, Lasse
Schwarten, Jens Teichert, Benjamin WaltherFranks. Multitouch Motion Capturing. ACM
International Conference on Interactive Tabletops
and Surfaces 2008, 2 pages, 2008.
[LNS05] Joe Laszlo, Michael Neff, Karan Singh.
Predictive Feedback for Interactive Control ofPhysics-based Characters. Computer Graphics
Forum (EUROGRAPHICS 2005), Vol. 24, No. 3,
pp. 257-265, 2005.
[LLZ*09] Xiubo Liang, Qilei Li, Xiang Zhang, Shun
Zhang, Weidong Geng, Performance-Driven

Motion Choreographing with Accelerometers,
Computer Animation and Virtual Worlds (CASA2009), Volume 20, Issue 2-3, pp. 89-99, 2009.
[MCC09] Jianyuan Min, Yen-Lin Chen, Jinxiang
Chai. Interactive Generation of Human
Animation with Deformable Motion Models.
ACM Transactions on Graphics, Vol. 29, Issue 1,
Article No. 9, 2009.
[Osh05] Masaki Oshita. Motion Control with Strokes.
Computer Animation and Virtual Worlds, Vol. 16,
Issues 3-4, pp. 237-244, 2005.
[Osh06] Masaki Oshita. Motion-Capture-Based
Avatar Control Framework in Third-Person View
VirtualEnvironments.
ACM
SIGCHI
International Conference on Advances in
Computer Entertainment Technology 2006 (ACE2006), 9 pages, 2006.
[PSS02] Sang Il Park, Hyun Joon Shin, Sung Yong
Shin. On-line locomotion generation based on
motion blending. ACM SIGGRAPH Symposiumon Computer Animation 2002, pp. 105-111, 2002.
[SL06] Hyun Joon Shin, Jehee Lee. Motion
Synthesis and Editing in Low-Dimensional
Spaces. Computer Animation and Virtual Worlds(CASA 2006), Volume 17, Issue 3-4, pp. 219227, 2006.
[TBvdP04] M. Thorne, D. Burke. M. van De Panne.
Motion Doodles: An Interface for Sketching
Character Motion. ACM Transactions ofGraphics (SIG-GRAPH 2004), Vol. 23, Issue 3,
pp. 424-431, 2004.
[Wel93] Chris Welman. Inverse kinematics andgeometric constraints for articulated figure
manipulation. M.Sc Thesis, Simon Fraser
University, 1993.
[ZMC*05] Victor B. Zordan, Anna Majkowska, BillChiu, Matthew Fast. Dynamic Response for
Motion Capture Animation. ACM Transactionsof Graphics (SIGGRAPH 2005), Vol. 24, Issue 3,
pp. 697-701, 2005.