Eurographics / ACM SIGGRAPH Symposium on Computer Animation (2004)

Posters and Demos

R. Boulic, D. K. Pai (Editors)

Pen-to-mime: A Pen-Based Interface for Interactive Controlof a Human Figure
Masaki Oshita
Kyushu Institute of Technology

Abstract
We introduce a pen-based intuitive interface to control a virtual human figure interactively. Recent commercial
pen devices can detect not only the pen positions but also the pressure and tilt of the pen. We utilize such information to make a human figure perform various types of motions in response to the pen movements manipulated bythe user. A figure walks, runs, turns and steps along the trajectory and speed of the pen. The figure also bends,
stretches and tilts in response to the tilt of the pen. Moreover, it ducks and jumps in response to the pen pressure.
Using this pen-based interface, the user controls a virtual human figure intuitively as if he or she were holding avirtual puppet and playing with it.

1 Introduction
There are many demands for an interactive motion control of a virtual character on desktop environments. However, this has been a difficult challenge. Since human figures have a large number of degrees of freedom (DOF) andtheir movements are complicated, it is not easy to control
them through a common device that has only a small number of DOF such as a mouse or gamepad. Although it ispossible to control complex motions by combining multiple
gamepads and/or mouse, the user needs a practice to learnsuch an interface design since the mapping from the multiple input devices to the movements of a controlled figure
may not be intuitive.
2 User interface
We introduce a pen-based intuitive interface to control avirtual human figure interactively. The key idea is that weuse a pen as a metaphor of a figure and map pen moments
to the figure motion (Figure 1). Recent commercial pen
devices can detect not only the pen positions but also thepressure and tilt of the pen. We utilize such information to
make a human figure perform various types of motions inresponse to the pen movements manipulated by the user. Afigure walks, runs, turns and steps along the trajectory andspeed of the pen. The figure also bends, stretches and tilts© The Eurographics Association 2004.

Figure 1: The pen-based interface. The figure movement
is associated with the pen manipulated by the use. The
positions, pressure, and tilt of the pen are used to makethe figure perform various motions.
in response to the tilt of the pen. Moreover, it ducks andjumps in response to the pen pressure. Using this pen-based
interface, the user controls a virtual human figure intuitively as if he or she were holding a virtual puppet andplaying with it (Figure 2).

M. Oshita / Pen-to-mime: A Pen-Based Interface for Interactive Control of a Human Figure

(a)

(b)

(c)

(d)

(e)

(f)

Figure 2: The pen-based motion control interface. On each image the left view shows the controlled figure and the rightview visualizes the movements of the pen manipulated by the user. The yellow stick shows pan tilt. The red arrow under thepen shows the orientation and velocity of the pen movements. The blue arrow shows the figure’s orientation. (a) The figure
walks and (b) runs in response to the pen movements. The figure turns and steps if the pens is moved in a different direction
with the figure orientation. (c) The figure also tilts, bends, and stretches in response to the pen tilt. (d) If the pen is pressed
onto the tablet for a while, the figure ducks in response to the pen pressure. (e) In addition, if the pen is pressed and released
quickly, the figure jumps. (f) The jump height and distance are determined based on the pen pressure and the locomotion
velocity when the pen is released. The user can make the figure perform various motions through an intuitive interface.
3 Motion generation
In order to generate varying motions based on multidimensional parameters that are given from a pen device(speed, angle, bend, tilt and duck), we take a motion
blending approach [PSS02] and construct motion blending
modules with a set of small number of motion capture data
for each type of motions: standing movements, locomotion,turns, steps, and jumps.
Motion parameters are computed based on the pen
movements on each animation step. The speed and angle
are computed from the velocity vector of the pen movements. Tilt, bend and stretch angles are simply computedfrom the pen tilt. Duck and jump heights are computed
using a low-pass and high-pass filter with the pen pressure.
On each step, one of motion modules is in charge of figure control. We also introduce a motion transition schemeand foot constraints for generating continuous motions.

On our preliminary experiments, although the users
found that our interface design was very interesting andenjoyed it, at the first time, they had some difficulties incontrol of multidimensional input with a pen. For example,
a user who tried to make the figure walk did not only movethe pen but also tilt or press it on the tablet unconsciously.
After five to ten minutes training, the users learned how tocontrol the multidimensional input independently.
We currently use relative movements of a pen for locomotion. We take this approach since it is important in ourinterface design that the figure movements is directly associated to the pen movements. This interface works wellwhen the user wants to the figure move in the specified
direction. However, it does not ensure a precise control oflocomotion trajectory. To realize both responsive and precise control, we need to introduce some control scheme
based on both absolute and relative positional information.
References

4 Experiments and discussion
We use a commercial pen and tablet device, intuos2 from
WACOM [Wac], and have implemented the proposed interface with the WACOM Tablet Library.

[PSS02][Wac]

PARK S. I., SHIN H. J., SHIN S. Y.: On-line Locomotion Generation Based on Motion Blending. In Proc. of ACM SIGGRAPH Symposiumon Computer Animation 2002, 113-120, 2002.
WACOM Tech. Co. http://www.wacom.com© The Eurographics Association 2004.