Character Motion Control by Hands and Principal Component Analysis
Masaki Oshita∗

Hayato Oshima

Yuta Senju

Syun Morishige

Kyushu Institute of Technology(a) Picture of our interface(b) Interface design

left hand: action control

right hand: pose control

blending weights ofprincipal components

blending weights ofprincipal components

action model selection action execution time

blending weights ofprincipal components

Figure 1: Proposed interface. (a) Our interface. (b) Interface design. To control a pose, the blending weights of the principal components arevaried by moving the right hand and bending/extending the fingers. To control an action, the primary key pose is altered by bending/extending
fingers in the same way as the pose control. An action is executed by moving the left hand forward and controlling the action execution time.
The current action model is selected by moving the left hand in the up-down and left-right directions.

Abstract

1 Introduction

In this paper, we propose an interactive character motion control interface that uses hands. Using their hands and fingers, the user cancontrol a large number of degrees-of-freedom (DOFs) at the sametime. We applied principal component analysis (PCA) to a set ofsample poses, and assigned the extracted principal components toeach DOF of the hands (such as the hand positions and finger bending/extending angles). The user can control the blending weights ofthe principal components and deform the character’s pose by moving their hands and bending/extending their fingers. We introducedpose and action controls, so that we can alter the standing pose andperform various actions with deformations. So that various typesof actions were possible, we constructed a number of action models in advance. We introduced action model selection and action
execution mechanisms. We developed methods for computing thefeature vector, for applying PCA, and for pose and action synthesis.
We present our experimental results and demonstrate the effectiveness of our interface.

The interactive control of character motion is important in real-time
applications such as computer games and communications using
avatars. However, character control in current applications is very
limited. Although there are various types of control devices suchas the game pad, keyboard, and touch screen, the user can only
select pre-defined actions by, e.g., pushing a button, touching thescreen, or performing a gesture. Users sometimes want to change
the pose and style of motions depending on the situation, but this is
not possible with conventional interfaces.

CR Categories: I.3.7 [Computer Graphics]: Three-Dimensional
Graphics and Realism—Animation; I.3.6 [Computer Graphics]:
Methodology and Techniques—Interaction techniques
Keywords: Motion Control, Hand, Principal Component Analysis, Computer Animation, User Interface∗ e-mail:oshita@ces.kyutech.ac.jp

In this paper, we propose an interactive character motion control
interface for hands. Using hands and fingers, the user can control alarge number of degrees-of-freedom (DOFs) at the same time. We
applied PCA to a set of example poses, and assigned the extracted
principal components to each DOF of the hands (such as the hand
positions and finger bending/extending angles). The user can control the blending weights of the principal components and deformthe character’s pose by moving hands and bending/extending fingers. We introduce pose and action controls for altering the standing pose and performing various actions with deformations. So
that various types of actions are possible, we constructed a numberof action models in advance. We introduced action model selection and action execution mechanisms. We developed methods forcomputing the feature vector, applying PCA and pose and action
synthesis. In this paper, we present our experimental results anddemonstrate the effectiveness of our interface.
Using fingers and hands, the user can make the character take various poses and perform actions with various deformations. This isan advantage of our method, when compared with conventional interfaces in current applications. Our interface provides an interesting experience of freely controlling a character’s pose and motion,using fingers and hands like a puppeteer.
The rest of this paper is organized as follows. Section 2 reviews
related work. Section 3 describes our interface and system design.
Section 4 explains our pose method, and Section 5 explains our action synthesis method. Section 6 presents our experimental results
and discussion. Finally, Section 7 concludes this paper.

2 Related Work
2.1 Principal Component Analysis (PCA)
PCA has been extensively applied to character motion processing.
In general, PCA is a popular technique for dimension reduction.
Because human poses and motions are high dimensional data, PCA
can be used to reduce the dimensions and make processing more
efficient. Safonova et al. [2004] applied PCA to increase the efficiency of computations in space-time motion optimization. Shinand Lee [2006] applied PCA to map example poses into a twodimensional latent space, so that the user can synthesize a pose ora motion by selecting a point or drawing a trajectory in the latent
space. Min et al. [2009] applied PCA to example motions instead ofposes, to synthesize new motions based on given constraints. PCA
is also often used for motion or gesture recognition [Ishigaki et al.
2009; Liang et al. 2009; Numaguchi et al. 2011]. However, thesemethods process the principal components internally, and do notprovide a way for controlling the blending weights. One reason forthis is that it is difficult to control multiple principal components
using conventional input devices. This research allows users to interactively control the multiple components using a hand motion
sensing device.
Crochow et al. [2004] considered another dimension reduction
technique, the scaled Gaussian process latent variable model (SGPLVM). They used it to map example poses onto a two- or threedimensional latent space for pose synthesis. Although SGPLVM ismore powerful than PCA, the latent space is nonlinear and its axes
do not represent specific pose deformations. Therefore, SGPLVM
does not fit with our interface, and we chose PCA.

2.2 Data Glove Interface
Researchers have previously used data gloves to control characters.
Okada [2003] introduced a metaphor of a puppet-like control and avirtual string. He mapped each finger, so that it represents the upward movement of a specific body part. Komura and Lam [2006]
controlled walking motions using a data glove. They mapped the
movements of two fingers to one-dimensional locomotion time. Oshita et al. [2013] also developed a puppet-like control, which usedboth hands and their fingers in the same way as an actual puppet.
These methods use the data glove to only control a part of the character’s body. This is because there are fewer DOFs for a data glove
than for a human pose. It is impossible to control the whole bodyusing these kinds of mappings.
Nik and Oshita [2012] used a data glove to specify parameters formotion blending [Rose et al. 1998; Park et al. 2002]. With theirinterface, the user only controls abstract parameters and cannot directly control the character’s pose.

2.3 Motion Control Interface
Various input devices are used as motion control interfaces.
There are many methods for controlling a character’s motion using
a conventional pointing device such as a mouse or pen. Generating locomotion along with a given trajectory is a common interface[Park et al. 2002]. However, with this type of interface, the type of
motion is limited to walking or running. Thorne et al. [2004] introduced gesture-based motion selection. Based on the gestures drawn
along a trajectory, their system inserts predefined motions such as ajump or flip. Oshita [2005] proposed a stroke-based motion selection technique that chooses an appropriate action according to theinitial and terminal points of a single stroke drawn on the screen.
With these systems [Thorne et al. 2004; Oshita 2005], users can

simply select actions by drawing a trajectory or stroke. However,
the executable motions are fixed and cannot be changed. Igarashi
et al. [2005] proposed a spatial keyframing animation technique.
This approach can continuously change a character’s pose based on
the cursor position. But to use this technique, the key poses must
be placed at appropriate positions that are dependent on the specificaction.
Multi-touch input devices such as tablet computers have recently
become widely available. There are some multi-touch interfaces
for interactive character motion control. Oshita [2012] applied astatistics-based inverse kinematics (style-based IK [Grochow et al.
2004]) to control the whole body of the character. However, thesynthesized poses are limited to a given set of example poses. To
utilize high dimensional inputs, a large number of example posesmust be given. Our interface can generate various poses using theprincipal components extracted from given example motions. Additionally, the combination of hand positions and finger angles has
more DOFs than multi-touch inputs.
Some researchers have combined physics simulations with controlling a limited number of DOFs. Laszlo et al. [2005] used a mouseto control a small number of joints, and generated the full-body motion using physics simulations. Shiratori and Hodgins [2008] used
Wii remotes to determine a few parameters for physics-based controllers of several actions such as walking, running, and jumping.
Physically plausible motion can be generated and controlled using
physics simulations. However, physics-based controllers must be
designed for each type of action in advance, and it can be hard toachieve various types of actions and styles.
Recently, low-price markerless motion capture devices such as Microsoft Kinect [Microsoft ] have become available. A character’s
full body motion can be freely controlled using these devices [Ishigaki et al. 2009; Oshita 2006], but they require a large workspace.
Moreover, the user must actually perform the highly dynamic actions, which is difficult for untrained users.

3 System Design
In this section, we describe our system design including input devices, interface design and system overview.

3.1 Input Devices
Our interface can work with any hand motion sensing device. In
this research, we used a depth-camera-based hand motion sensing
solution, which combines a PrimeSense camera [PrimeSense ] (asused in a Microsoft Kinect [Microsoft ]) and a third party hand motion sensing library [3Gear Systems ]. Depth-camera-based technologies have recently been extensively applied [PrimeSense ; Leap
Motion ; Intel ]. They have the advantage that the user does not haveto wear a device. Alternatively, we can use a device such as a data
glove [5DT ] to capture finger movements, and magnetic [Polhemus] or optical tracking devices [Natural Point ] to capture hand movements. Using these hand motion sensing devices, the positions andorientations of two hands and the bending/extending angles of ten
fingers (or positions of finger tips) can be measured in real time.
Our method uses only the hand information given in Table 1.
All input data from the hand motion sensing device (Table 1) arenormalized between −1 ∼ 1. When a finger is relaxed and slightly
bent, its finger angle is zero. We allow some variation in the zero
value range. When the finger is completely bent, the angle is 1.
When the finger is fully extended, the angle is −1. The initial position of a hand is (0, 0, 0), and it can vary between (−1, −1, −1)and (1, 1, 1). The initial position and operational range are acquiredTable 1: Input parameters from the hand motion sensing device.
Parameter
finger angles (both hands)
hand positions (both hands)(a) During pose control

Dimension
5 × 2 = 10
3×2=6Table 2: Control parameters in our method.
Parameter
blending weights ofprincipal components
for pose control
blending weights ofprincipal components
for primary key pose
for action control
action execution time
action model selection

Variable

Dimensionθ6(up to 8)

φts3(up to 5)
12(b) During action control

for each hand during a calibration process, and the input positions
are scaled during run time. We also apply a low-pass filter to thefinger angles and hand positions to reduce noise. In our implementation, we apply a moving average filter, which calculates theaverage of n data series.

3.2 Interface Design
The interface design of our method is shown in Figure 1. We haveintroduced pose and action controls. The right hand is used to control the pose, and the left is used to control the action.
To control the pose, we applied PCA to a set of example standing
poses and assigned the extracted principal component to each DOF
of the right hand (hand position and finger bending/extending angles). When the hand is in its initial position and all the fingers
are relaxed, the character has a neutral standing pose. By moving the right hand and bending/extending its fingers, the blending
weights of the principal components are controlled and the synthesized standing pose changes dynamically. Among the extracted
principal components, the one that creates the largest movement of
the center of mass in the x-axis direction was assigned to the x-axis
position of the user’s right hand. The same mapping was applied to
the y- and z-axes positions. The other principal components withlarger movements were assigned to the fingers of the right hand.
This way the user can intuitively remember the mappings between
the DOFs and principal components.
To control the action, we applied PCA to a set of primary key
poses, using example motions for each type of action. The primary key pose is the most important (representative) key pose in
the action (e.g., for the punching action, the pose with the arm extended the most). The primary key poses were manually specified
in advance. The principal components extracted from the primary
key poses were assigned to the finger bending/extending angles ofthe left hand. By bending/extending the fingers of the left hand,the user can control the blending weights of the principal components and alter the primary key pose of the action, in the same way
as the pose control. The controlled key pose is visualized on thescreen as shown in Figure 2. By moving the left hand forward, example motions are blended to realize the controlled key pose, andthe synthesized action is performed. It is also possible to adjust
the primary key pose by controlling the weights of the principal
components when the action is being executed. We chose to use
the primary key pose for action control, because it is important torepresent the style of the action when it being executed . And it is

Figure 2: Screen shots of our prototype.

impossible to control multiple poses, even with a large number of
DOFs.
So that we can control various types of actions, a number of action
models were constructed in advance. The current action model isselected by moving the left hand in the up-down and right-left directions. In general, the principal components depend on the type ofaction; PCA must be applied to a set of example poses for the same
type of action. Therefore, separate action models were constructed
for each type of action. Each action model was mapped into themodel selection space in advance. The current action model is selected using the position of the left hand.
The control parameters for our method are summarized in Table 2.
Although it is possible to capture the bending/extending angles offive fingers for each hand, we used only three fingers (the index,
middle and ring fingers). This is because it is difficult to movethe little finger independently from the ring finger, and it is hard tomeasure the movement of the thumb using the depth-camera.
Figure 2 contains screen shots of our prototype. The controlled parameters and the model selection space are shown on the left andright side on the screen for monitoring. However, this information
is not necessary when using our interface. There are six models in
the model selection space. The blue dot represents the current position in this space. When an action model is selected, the primary
key pose synthesized from the current blending weights is visualized on the screen as a red stick figure.
As an alternative mouse-based interface, the user can control theparameters by dragging the sliders, select the model selection parameter by clicking a point on the model selection space, and exe-

cute the action by double clicking. We compared our hand motion
interface with this alternative mouse-based interface in our experiments.

3.3 System Overview
Our system consists of two stages: set up and run time. During
the set up process, one pose control model and a number of action
control models are constructed by applying PCA to example poses
from given example motions. A pose control model is constructedfrom a number of example standing poses. As explained above, weconstruct separate models for each type of action. For constructing
an action control model, we need a number of example motions andtheir timings for the primary key pose. An example motion can beeither a motion capture sequence (a series of poses) or a keyframe
animation.
In our experiments, we constructed six action control models:
punch, kick, avoid, guard, jump, and walk. Each action model
is constructed using two to five example motions. The pose control model is constructed using approximately 15 example standing
poses.

local coordinates
initial position of
the base foot(origin)initial orientationof the body
(z-axis)Figure 3: Feature vector representation. The positions and orientations are represented by the local coordinates which are definedby the initial position of the base foot and the initial orientation of
the body. A blue rectangle represents a position and a read circle
represents a rotation (orientation).

During run-time, the control parameters in Table 2 are computed
based on the input from the device in Table 1 and are then used tocontrol the poses and actions.

it is important for our method to control poses and realize actions
that include global translations and rotations such as walk, step, andjump.

4 Pose Control

We use rotational angles (Euler angles) for the joint rotations, because other representations (such as matrix and quaternion) cannot
be represented by a linear combination of the principal components.

In this section, we describe our pose synthesis method, which
blends the principal components extracted from a set of example
standing poses.

4.1 Feature Vector
To apply PCA to a set of example poses, each example pose must
be represented as a feature vector. The definition of a feature vector
is important to PCA and the blending of the principal components.
In general, a character’s pose is represented by the position and orientation of the root segment (pelvis) and the rotations of all joints.
There are several ways to represent joint rotation such as combinations of rotational angles (Euler angles) or a 3 by 3 rotational
matrix and quaternion. However, this conventional representation
is not suitable for PCA, because independently blending joint rotations can unnaturally change the position of the end-effector (e.g.,foot).
In this research, the feature vector consists of the position and orientation of the pelvis, positions and orientations of both feet, swivel
angles of the knees, and joint rotational angles for all joints in theupper body. The positions and orientations are represented in thelocal coordinates of the example action, defined by the position of
the base foot and the orientation of the body (as shown in Figure 3).
The joint rotations are represented by their rotational angles.
The local coordinates of the example motion are defined so that its
origin matches the position of the base foot, and its z-axis matches
the body orientation. The body orientation represents the direction that the character is facing, and is sometimes different fromthe pelvis direction [Oshita 2008]. The base foot and body orientation are specified manually for each example motion. The y-axis
is always the up vector, and the x-axis is the cross product of the zand y-axes. We use these local coordinates to keep the positions of
the feet during posture synthesis. Previous work that applied PCA
to character poses [Shin and Lee 2006; Ishigaki et al. 2009; Lianget al. 2009; Numaguchi et al. 2011] discarded the global positionand orientation of poses and processed them separately. However,A leg pose is represented by the position and orientation of the feet,and the swivel angle of the knee. The pose is reconstructed fromthe feature vector using analytical inverse kinematics [Tolani et al.
2000]. For arm poses, it is possible to use either the position ororientation of the hand, or the joint angles in the arm. We tested
both implementations, and decided to use the joint angles, because
position of the hand did not produce well synthesized poses. We
believe that this is because the global hand positions are influenced
by the whole body and the arm postures, and the relationships arestrongly nonlinear.
In total, the feature vector has 52 DOFs.

4.2 Application of PCA
To apply PCA to a set of feature vectors fi , we first computed theiraverage feature vector faverage . Then, PCA was applied to the differences between the feature vectors and the average feature vector
f i = fi − faverage .
The PCA is computed as follows. We first calculate the covariancematrix C of all the feature vectors f i . C is a n × n matrix, where
n is the dimension of the feature vector. Then, we compute theeigenvectors and eigenvalues of the covariance matrix C. They are
P−1 CP = D(1)

The matrix P consists of the eigenvectors pj , which represent the
principal components. The matrix D is the diagonal matrix ofeigenvalues λj , which represent the contributions of the principal
components. Both P and D are n × n matrices. Because we have
six DOFs (three for hand position and three for finger angles) tocontrol the pose, we can control the weights of the six most influential principal components from the n principal components in P.
It is known that each kind of action can be effectively reproduced
using between four and seven principal components [Safonova et al.
2004]. It is considered that it is sufficient to control approximately

six principal components. In our implementation, Equation 1 iscomputed using the Jacobi eigenvalue algorithm [Press et al. 1992].

4.3 Pose Synthesis

Cj = C(f +j ) − C(f −j )The feature vector of an output posture f is synthesized by blending
the principal components with given weights w. That is,
f = wt P + faverage(2)

As explained in Section 3.1, the input DOF θj is such that the hand
position (j = 1 ∼ 3) and finger angles (j = 4 ∼ 6) are normalized between −1 ∼ 1. The appropriate scale for each principal
component varies. To determine them, we first calculate the sets ofweights for realizing each example pose using Equation (2). We use
the maximum and minimum weights wmax,j , wmin,j to determine
the range and scale of each principal component. That is,wj =wmax,j θj−wmin,j θjmass of a pose f . The maximum displacement of the center of massfor each principal component is(if θj ≥ 0)
(if θj < 0)(3)

As result, the output feature vector f is computed from θj (j = 1 ∼6) using Equations (2)–(3). When all the fingers are relaxed, thepose from the average feature vector is synthesized. By moving thehand and bending/extending the fingers, the corresponding principal component is added to the synthesized pose. This allows theuser to interactively change the pose.
The character’s pose is reconstructed from the output feature vector f , using the coordinates computed from the character’s current
position and orientation. The character’s initial coordinates (initial
position and orientation) are given first. They are updated as thecharacter performs actions. For example, when the character walks,
steps or jumps, the character’s coordinates are updated accordingly.
Some previous methods that used a low-dimensional latent spacesuch as PCA for pose synthesis [Min et al. 2009; Grochow
et al. 2004] introduced additional post-processing for correcting theweights. This is because a combination of weights may produce anunnatural pose. However, our method does not use post-processing,
because it would limit the synthesized pose to the given example
poses. The purpose of our method is to synthesize new poses byadding some deviations (principal components), rather than choosing an existing example pose. If many example poses that include
all possible deviations can be prepared, it may be effective to limit
the synthesized pose within the example poses. However, this is
not practical. Moreover, because the pose modification caused byeach principal component is a linear deformation that the user canvisualize during control, they can avoid unexpected poses by themselves.
We also did not introduce additional processes to maintain balance,
for the same reasons. However, unbalanced poses are sometimes
generated. There are various methods for changing a character posefor balancing [Macchietto et al. 2009; Jain et al. 2009]. However,
it is difficult to introduce these methods into an interactive posecontrol system. Although this may be an interesting extension tothis work, it is outside of the scope of this paper.

4.4 Assignment of Principal Components
As explained in Section 3.2, the principal components are assigned
to each DOF based on the magnitude of their influence on the posedeformations. Let C(f ) be the function that computes the center of(4)

where f +j and f −j are computed using θi+j = 1(i = j), 0(i = j)and θi−j = −1(i = j), 0(i = j), respectively. Among the extracted principal components, the one that results in the largest
movement of the center of mass Cj in the x-axis direction is assigned to the x-axis position of the user’s right hand. The same
mapping is applied to the y- and z-axes positions. Among the rest of
the principal components, the three that represent the largest movements are assigned to three fingers of the right hand. Because there
is no criterion for this mapping, we simply assign the three principal
components from the index finger to the ring finger.

5 Action Control
Action control model is constructed for each type of action. The
primary key pose is synthesized by using the same method with thepose synthesis. Action synthesis is done by blending the example
motions to realize the controlled primary key pose.

5.1 Application of PCA
A number of example motions must be given to construct an action
model. The timing for the primary key pose must be specified foreach example motion. PCA is applied to the primary key poses.
The extracted principal components are assigned to the fingers of
the left hand. In the same way as pose control, we use the principal components that represent larger movements of the center ofmass. By bending and extending the fingers, the user can deformthe primary key pose.

5.2 Action Synthesis
During action synthesis, the character’s pose (feature vector) is
computed by blending the poses (feature vectors) from the example
motions with the weights x determined by the controlled primary
key pose.
f (t) =m

xi fi (t)

(5)

i=1

where fi is the pose (feature vector) from i-th example motion at
the normalized time t, and m is the number of example motions.
Let wi be the weights of the principal components for generating
the i-th key pose. The relationship between the weights of examples
x and the weights of the principal components w is represented by{w0 w1 ...wn }x = w(6)

Wx = w(7)

Only the controllable elements are considered for the weights ofthe principal components. Therefore, the weights (n ) have three
dimensions. Assuming that m > n , the weights of examples x
are computed from the weights of principal components w, using a
pseudo inverse matrix W+ . That is,
W+ w = x(8)Because W is constant, W+ is only computed once.
When an action starts and finishes, there is a transition between
pose and action synthesis. To realize smooth transitions, we must
blend two poses from the pose and action synthesis.

5.3 Action Execution
The action execution time is controlled by moving the user’s handforward. When the forward-backward hand position (z-axis) is between −1 ∼ 0, the action time is zero and the action is not performed. When it is between 0 ∼ 1, the action time is controlled
and the action is executed. The z position represents the normalized action time t between 0 ∼ 1, where t = 0 is the initial poseof the action and t = 1 is the terminal pose. During execution,
the pose at time t is generated by blending example motions using
Equation (5).
However, if the example pose at a given normalized time t is simply
synthesized, if the hand moves quickly, the action may be executed
too quickly and look unnatural. Because the duration of the action
varies depending on the type of action, it is difficult for the user toknow the appropriate speed for the hand movement. Therefore, thetime specified by the user’s input is used as the target time. The
execution time is advanced or reversed toward the target time at areasonable speed.
The user can normally control the execution time by using theirhand position to advance or reverse it. However, at certain points inan action, reversing will appear unnatural. For example, consider akick. It is possible first lift the foot up and down. However, after
the kick action starts, it is difficult to pull the foot back. To prevent
this, we specified the time of no return for each action in advance.
After the action execution time passes this time, it is automaticallyadvanced and cannot be changed by the user.
As explained in Section 5.2, during the initial and terminal periodsof an action, the output poses from the pose and action controls
are blended to create a smooth transition. As explained in Section
4.1, the character’s coordinates are updated when the action is completed.

5.4 Action Model Selection
The user can select the current action model from a number of prepared action models. The action models are placed on the model
selection space (two-dimensional space). The user can specify aposition on the space s = {s0 , s1 } using the x and y (left-right andup-down) hand positions. The action model is selected based on thespecified position.
We assigned a radial basis function (RBF) to each action model.
RBFs are commonly used for computing blending weights fromgiven parameters [Park et al. 2002]. The distances from each action
model dk are computed using the RBF. The radius of an RBF canbe specified manually or automatically tuned based on the distanceto the nearby action models. The distance from an action model iscomputed using
dk = R(s) = B(|s − sk |/rk )(9)where sk and rk represent the position and radius for each action
model, respectively. The action model with the largest distance (dk )is selected as the current action model.
When the action is being executed, the system does not allowchanges to the current action model. When a different model isselected, the synthesized poses from the action execution and theselected model are blended to create a smooth transition during theterminal period of the execution.

6 Results and Discussion
In this section, we present our experimental results and some discussion. The accompanying video includes a demonstration of ourinterface. Examples of the principal components are shown in Figure 4 and 5.

6.1 Evaluation
We asked six undergraduate and graduate students in computer science to participate in our evaluation. They have a basic knowledgeof computer animation. We also exhibited our system at a university event, and demonstrated it to over 100 visitors (high school
students and parents). Some of them observed the demonstration,and some tried the system for a short time. We collected feedback
from all these sessions.
Basically, the participants recognized that our interface has theunique advantage that the user can deform the character pose using their fingers. However, they also commented that some training
would be required before using the interface.
The effectiveness of our interface for controlling poses depends
on the extracted principal components. To use our interface, theuser must remember the mappings between each DOF and the corresponding pose deformation. As shown in Figure 4, in our experiments, effective principal components were extracted and werealized an intuitive mapping. In this case, the principal component for the x position also included the movements along the zaxis, because there were no principal components that only represented movements along the x-axis. Nevertheless, the three principal components for the hand position approximately represented
movements in each direction, and we considered that the mapping
was intuitive. For the three principal components represented byfinger angles, one was used to open/close the arms, one adjusted thefoot and pelvis positions, and one rotated the back. These were also
considered effective for altering the standing pose. Figure 5 shows
the principal components for the punch action control model. We
cannot associate them with positions (directions) in the action control. However, because we extracted the effective principal components and there are three principal components, the user can easily
understand how to alter the primary key pose of the action using theprincipal components.
We found that fingers resulted in an intuitive control interface, butwere inaccurate because of problems in the sensors and the abilitiesof the users. It is difficult to independently operate multiple fingers.
When a finger is moved, there are small unintentional movements inthe adjacent fingers. This limitation sometimes caused unexpected
pose deformations. Some finger movement combinations are difficult to perform, for example, bending one finger and extending thenext. We also noticed that it is difficult to keep fingers at certain angles. It is easy to move the hands to select the models and executeactions. However, the hand can sometimes move unintentionally,
which changes the primary key pose. It was a little difficult to keep
the hand still, and this created fatigue in the arms.
In our experiments, the system has six action models. Our framework can handle any number of action models, although we did
not test this. However, if there are too many action models, it isdifficult to select one from the model selection space because therecognizable range of hand positions is limited.

average pose

principal components #1~6
1 (x position)
2 (y position)

3 (z position)

4 (index angle)

5 (mid. angle)

6 (ring angle)

Figure 4: An example of the principal components for the pose control model. The poses from the maximum and minimum weights of eachprincipal component are synthesized.

average pose

principal components #1~3
1 (index angle)

2 (mid. angle)

3 (ring angle)

Figure 5: An example of the principal components for the punch action control model. The primary key poses from the maximum andminimum weights of each principal component are synthesized.

6.2 Comparison
We compared our interface with the alternative mouse-based interface using sliders, as described in Section 3.2. If the user simply
wants to select an action model from the model selection space andexecute an action, the mouse-based interface was much easier touse. However, it is obviously difficult to control multiple weights atthe same time, and interactively deform the character pose. Thereis a trade-off between the DOFs and ease of use.

and action synthesis methods. Some target poses were generated
from the pose control model, whereas some target poses were generated from the primary key poses of the action control models. We
used random weights for one to three DOFs, and zero weights forthe other DOFs. The results were measured for each type of target
pose. When using our interface, if the primary key pose of an action
was displayed as the target pose, the subject needed to execute an
action after selecting an appropriate action model and controlling
the primary key pose. When the pose editing interface was used,
there were no differences between the target poses from the poseand action control models. We recorded the time required to reacheach target pose from the initial pose. Each subject used all threeinterfaces in different orders, after a short training period.

We conducted a user test to show the efficiency of our interface
compared with a conventional interface. Because there is no alternative common interface that can interactively control a character’s
actions with deformations, it is difficult to compare our interface foraction control tasks. Therefore, we evaluated our interface for posing control tasks. We compared our interface using hands or mouse(using sliders) with a conventional mouse-based pose editing interface. In this conventional technique, a joint rotation or position canbe manipulated using handles displayed on the screen, as shownin Figure 6. The blue and red spheres on the character represent
controllable end-effectors and joints, respectively. This kind of interface is commonly used in commercial animation systems such as
Maya, 3ds Max, and Softimage . Although only one body part canbe controlled at a time, its position and rotation can be controlled
precisely.

The average times for the posing tasks are shown in Table 3. For all
target poses, our hand interface was more efficient than our mouseinterface and the pose editing interface. These results indicate thatprincipal components are an efficient way of controlling poses witha given set of sample poses. They also demonstrate that hands andfingers can be used to effectively control the weights of the principal
components, and are more efficient than a mouse and sliders.

The six subjects described in Section 6.1 participated in the user
test. During the experiment, the subject was shown a target poseand he or she was asked to make the same pose using one of the interfaces. The system determines that the task is completed when the
maximum distance between the positions of all joints in the target
and controlled poses is below a threshold (10 cm). Different typesof target poses were generated with random weights using our poseIn this paper, we proposed an interactive motion control interface
based on hands. Using their hands and fingers, the user can interactively deform the character’s pose by controlling the blending
weights of the principal components. The user can execute various
actions by altering the primary key pose. Although our interface
requires some training, the user can control the character in a waythat is not possible using conventional interfaces.

7 Conclusion

L EAP M OTION. http://www.leapmotion.com/.
L IANG , X., L I , Q., Z HANG , X., Z HANG , S., AND G ENG , W.
2009. Performance-driven motion choreographing with accelerometers. Computer Animation and Virtual Worlds (CASA2009) 20, 2ʖ 3, 89–99.
M ACCHIETTO , A., Z ORDAN , V., AND S HELTON , C. R. 2009.
Momentum control for balance. ACM Transactions of Graphics(SIGGRAPH 2009) 28, 3, Article No. 80.

Figure 6: Mouse-based interface for comparison. The position andorientation of a joint or end-effector can be controlled by dragging
a handle.

M ICROSOFT. Kinect. http://www.xbox.com/en-US/kinect.

Table 3: Results from the user test. Average time for making eachtarget pose.

M IN , J., C HEN , Y.-L., AND C HAI , J. 2009. Interactive generation of human animation with deformable motion models. ACM
Transactions on Graphics 29, 1, Article No. 9.

target typepose (1 DOF)
pose (2 DOF)
pose (3 DOF)
action (1 DOF)
action (2 DOF)

our interface(hand)
15.0 s22.3 s19.2 s28.0 s30.5 sour interface(mouse)19.7 s
22.4 s24.6 s32.4 s36.5 spose editing(mouse)95.1 s63.9 s
95.0 s
98.0 s138.9 sNew technologies for sensing hand motions [Leap Motion ; Intel ]are becoming common. We believe that our interface can be a good
application of these technologies.

Acknowledgment
This work was supported in part by a Grant-in-Aid for Scientific
Research (No. 24500238) from the Japan Society for the Promotionof Science (JSPS).

References
3G EAR S YSTEMS. 3gear devkit. http://www.threegear.com/.
5DT. Data-glove. http://www.5dt.com/.
G ROCHOW, K., M ARTIN , S. L., H ERTZMANN , A., AND
P OPOVI Ć , Z. 2004. Style-based inverse kinematics. ACM Transactions on Graphics 23, 3, 522–531.
I GARASHI , T., M OSCOVICH , T., AND H UGHES , J. F. 2005.
Spatial keyframing for performance-driven animation. In ACM
SIGGRAPH / Eurographics Symposium on Computer Animation
2005, 253–258.
I NTEL. Perceptual computing sdk. http://software.intel.com/enus/vcsource/tools/perceptual-computing-sdk.
I SHIGAKI , S., W HITE , T., Z ORDAN , V., AND L IU , C. K. 2009.
Performance-based control interface for character animation.
ACM Transactions of Graphics (SIGGRAPH 2009) 28, 3, Article No. 61.
JAIN , S., Y E , Y., AND L IU , C. K. 2009. Optimization-based
interactive motion synthesis. ACM Transactions on Graphics
28, 1, Article No. 10.
KOMURA , T., AND L AM , W. C. 2006. Real-time locomotion control by sensing gloves. Computer Animation and Virtual Worlds
17, 5, 513–525.
L ASZLO , J., N EFF , M., AND S INGH , K. 2005. Predictive feedbackfor interactive control of physics-based characters. Computer
Graphics Forum (EUROGRAPHICS 2005) 24, 3, 257–265.

NATURAL P OINT. Optitrack. http://www.naturalpoint.com/.
N IK , I. N. I., AND O SHITA , M. 2012. Evaluation of a dataglove-based interface for motion selection and style control.
In IIEEJ Image Electronics and Visual Computing Workshop(IEVC) 2012), 6.
N UMAGUCHI , N., NAKAZAWA , A., S HIRATORI , T., AND H OD GINS , J. K. 2011. A puppet interface for retrieval of motion
capture data. In ACM SIGGRAPH/Eurographics Symposium on
Computer Animation 2011, 157–166.
O KADA , Y. 2003. Real-time motion generation of articulated figures using puppet/marionette metaphor for interactive animation
systems. In 3rd IASTED International Conference on Visualization, Imaging, and Image Processing (VIIP03), 13–18.
O SHITA , M., S ENJU , Y., AND M ORISHIGE , S. 2013. Character motion control interface with hand manipulation inspiredby puppet mechanism. In 12th ACM SIGGRAPH International
Conference on Virtual-Reality Continuum and its Applications in
Industry (VRCAI) 2013, 131–138.
O SHITA , M. 2005. Motion control with strokes. Computer Animation and Virtual Worlds 16, 3-4, 237–244.
O SHITA , M. 2006. Motion-capture-based avatar control frameworkin third-person view virtual environments. In ACM SIGCHI International Conference on Advances in Computer Entertainment
Technology (ACE) 2006, Article No. 2.
O SHITA , M. 2008. Smart motion synthesis. Computer Graphics
Forum (Pacific Graphics 2008) 27, 7, 1909–1918.
O SHITA , M. 2012. Multi-touch interface for character motion
control using example-based posture synthesis. In International
Conference on Computer Graphics, Visualization and Computer
Vision (WSCG) 2012, 213–222.
PARK , S. I., S HIN , H. J., AND S HIN , S. Y. 2002. On-line locomotion generation based on motion blending. In ACM SIGGRAPH
Symposium on Computer Animation 2002, 105–111.
P OLHEMUS. Fastrak. www.polhemus.com.
P RESS , W. H., T EUKOLSKY, S. A., V ETTERLING , W. T., AND
F LANNERY, B. P. 1992. Numerical recipes in C (2nd ed.): theart of scientific computing. Cambridge University Press, New
York, NY, USA.
P RIME S ENSE. Carmine. http://www.primesense.com/.
ROSE , C., C OHEN , M. F., AND B ODENHEIMER , B. 1998. Verbs
and adverbs: Multidimensional motion interpolation. IEEE
Computer Graphics and Applications 18, 5, 32–40.

S AFONOVA , A., H ODGINS , J. K., AND P OLLARD , N. S.
2004. Synthesizing physically realistic human motion in lowdimensional, behavior-specific spaces. ACM Transactions on
Graphics (SIGGRAPH 2004) 23, 3, 514–521.
S HIN , H. J., AND L EE , J. 2006. Motion synthesis and editing
in low-dimensional spaces. Computer Animation and Virtual
Worlds 17, 3-4, 219–227.
S HIRATORI , T., AND H ODGINS , J. K. 2008. Accelerometer-baseduser interfaces for the control of a physically simulated character.
ACM Transactions on Graphics (SIGGRAPH Asia 2008) 27, 5,
Article No. 123.
T HORNE , M., B URKE , D., AND VAN DE PANNE , M. 2004. Motion doodles: An interface for sketching character motion. ACM
Transactions of Graphics (SIGGRAPH 2004) 23, 3, 424–431.
T OLANI , D., G OSWAMI , A., AND BADLER , N. I. 2000. Realtime inverse kinematics techniques for anthropomorphic limbs.
Graphical Models and Image Processing 62, 5, 353–388.