A tangible interface using 3D printed ﬁgures forsearching for combat motions of two characters
Kouhei Hino

Masaki Osita

Kyushu Institute of Technology
Iizuka, Fukuoka, Japan
p236054k@mail.kyutech.jp

Kyushu Institute of Technology
Iizuka, Fukuoka, Japanoshita@ces.kyutech.ac.jp

Abstract—In this paper, we propose a tangible interface using
three-dimensional (3D) printed ﬁgures to search for the combat
motion of two characters, which consist of an attacking motionof one character and the associated defending or avoiding motionof the other character. A key idea is that such combat motion
can be identiﬁed by the key poses of two characters and spatial
relationship between them at the moment of the impact of theattack. Another key idea is that the poses and their spatial
relationship can be speciﬁed for searching for combat motions
by arranging 3D ﬁgures of the key poses on a stage. A userof our system can easily search for the combat motion of two
characters from a motion database by choosing and arranging
the 3D ﬁgures. We developed a method for searching for thedesired pair of motion from the motion database based on user
input. We implemented a prototype to evaluate our approach andconducted a user study to show the effectiveness of our interface
by comparing it with a conventional graphical user interface for
specifying the same information.
Index Terms—Motion retrieval, 3D printed ﬁgures, tangibleinterface, combat motion

Fig. 1. Keypose of an attacking motion. The moment of impact is used as
keyframe.

I. I NTRODUCTION
In recent years, three-dimensional (3D) computer animation
has been used in many types of digital content, such as movies.
Motion data are essential for creating 3D computer animation.
Because it is difﬁcult to create motion data, there is a demand
for reusing existing motion data. However, as motion data arerepresented by the time-varying joint rotation of an articulated
ﬁgure and are complicated, it is difﬁcult to search them using
a keyword or any other input. Furthermore, even though theassociated motion of two or more people, such as combat
motion, are often used in computer animation, searching forsuch associated motion of multiple people is more difﬁcult
than searching for the motion of a single person because not
only must the motion of each person be taken into accountbut also the spatial relationship between the motion.
In this study, we develop a tangible interface using 3D
printed ﬁgures to search for the combat motion of two characters, which consist of an attacking motion of one characterand the associated defending or avoiding motion of the othercharacter. A key idea is that such combat motion can beidentiﬁed by the key poses of two characters and spatial
relationship between them at the timing of the impact of theattack, as shown in Fig. 1. Another key idea is that a queryof the poses and their spatial relationship can be given byarranging 3D ﬁgures of key poses on a stage, as shown inFig. 2. Created ﬁgures (a) and the proposed interface (b).

Fig. 2. A user of our system can easily search for combat
motion of two characters from a motion database by choosing
and arranging 3D ﬁgures. Using 3D printed ﬁgures, the user
can choose key poses available in the database. Additionally,
the user can specify the spatial relationship between ﬁguresphysically without using a complex graphical user interface.
Even a user who is not familiar with conventional animation
systems, such as a movie director, choreographer or animator,
can use our system to search for existing combat motion.

Our system uses a motion database, which contains thecombat motion of two characters. The key timing for eachpair of motion is speciﬁed in advance. The 3D ﬁgures ofkey poses of these motion are created using a 3D printer.
We develop a method for searching for a desired pair ofmotion from the motion database based on user input. We
deﬁne combat motion features that contains the poses of two
characters, and the relative position and orientation between
them. Based on those features, we also design an evaluation
function to evaluate the similarity between user input and eachpair of combat motion. We conducted a user experiment todemonstrate the effectiveness of our interface by comparing
it with a conventional graphical user interface that speciﬁes
the same information. The results demonstrated that, for ourinterface, the operation time was shorter and the number oftrials was smaller than those of a conventional interface.
II. R ELATED R ESEARCH
Various interfaces have been developed to search for motion.
A basic approach is to specify a motion name and searchusing a keyword [1]. However, using this approach, it is
difﬁcult to specify the spatial relationship between poses. Aninterface was proposed that uses the user’s pose acquired
using Microsoft Kinect to search for motion [2]. This method
requires a large space. Another interface, which uses a doll,
was proposed by Numaguchi et al. [3]. The user performs thesearch by moving the joints of the doll and reproducing the intended motion. However, all these methods cannot specify themotion of more than one person and their spatial relationshipsimultaneously. Won et al. [4] proposed a method to search fora series of motion of more than two people based on an event
graph. However, using this method, it is difﬁcult to specify aspatial relationship. Our interface allows the user to specify
the key poses and spatial relationship of two characters using
physical 3D ﬁgures.
III. M OTION DATABASE AND 3D FIGURESIn this section, we describe the motion database and 3Dﬁgures that are created in advance.
A. Motion database
As explained in Section 1, our motion database contains a
number of combat motion of two characters, which consistof an attacking motion of one character and the associateddefending or avoiding motion of the other character. We
assume that these combat motion are created in advanceusing motion capture or keyframe animation. The motion dataconsist of the body model and time-varying poses of thecharacter. In this study, we assume that all motion have asimilar standard body model; hence, we do not consider thedifference between body models. A pose is represented byjoint rotation, and the position and orientation of the pelvis.
The key frame for each pair of motion is speciﬁed inadvance. The key frame is the timing of the impact of theattack, where the limb of the end-effector is moved furthestfrom the initial posture. Although there are methods for

detecting key poses from motion automatically [5], it is still
difﬁcult to determine the timing of impact. Therefore, in thisstudy, we manually specify the key frames for each pair ofcombat motion by visually inspecting them.
For our experiment, we created 60 pairs of combat motion.
Our motion database contained four types of basic motion, thatis, two attacking motion of punch and kick, and two defending
motion of duck and guard. We created 12 combinations of
the relative position and orientation for each pair of attacking
and defending motion. The combinations included two typesof relative distances, which were far or near (approximately0.9 m and 0.45 m, respectively), and three types of relativeorientations, which were facing each other, side or back(approximately 0◦ , 90◦ and 180◦ , respectively) These motion
were created using a motion capture device [6]. We manuallyspeciﬁed the key frames for all pairs of motion, such as thoseshown in Fig. 1.
B. 3D ﬁgures
A 3D ﬁgure was created based on the pose in the key frameof a combat motion. Because the motion data did not containa shape model, we prepared a deformable shape model that
was associated with the standard body model in our motion
data. The shape of the 3D ﬁgure was generated by deforming
the shape model according to the key pose. Additionally, thebase of the ﬁgure was separately generated so that it covered
the positions of the feet on the ground and the center of mass.
The ﬁgure was scaled to 1/16 of the size of an actual humanﬁgure. We used a 3D printer to create physical 3D ﬁgures. The
colors of the ﬁgures are separated for attacking and defending
motions.
Because it was not practical to create 3D ﬁgures for all key
poses of the combat motion in the database, we classiﬁed all
key poses into groups of similar poses and created a representative ﬁgure for each group. The similarity between key posesof attacking motion was measured from the end-effector for
attacking and its height. The similarity between key poses ofdefending motion was measured based on whether the motion
was guard or avoid, and in the case of guard, from the endeffector for guarding that was in contact with the attacker andits height.
In our experiment, because our motion data contained only
four types of motion, as described in Section III.A, we createdfour 3D ﬁgures, as shown in Fig. 2 (a).
IV. I NTERFACE D ESIGN
The interface of our system is shown Fig. 2 (b). First, theuser selects one ﬁgure from the group of attacking ﬁgures
and another ﬁgure from the group of defending ﬁgures. The
user then places them on the transparent platform. Our systemsearches for the pair of combat motion that matches the user’s
query the most from the database and displays the animationon the screen. The user repeats this process until a satisfying
result is found.
To detect the types of ﬁgures and their placement on theplatform, our system uses AR markers, which are placed on theFig. 3. AR marker on the base of 3D ﬁgure.

base of the ﬁgure, as shown Fig. 3. The ID numbers, positions
and orientations of the markers are recognized from the image
from the camera that is placed under the transparent platformusing the ID numbers that are assigned to the 3D ﬁgures inadvance. The recognition of AR markers will be implemented
using the ARToolKit software library [7]; it is not implemented
in our system yet.
V. M ETHOD FOR SEARCHING FOR COMBAT MOTION
In this section, we describe our method for searching forcombat motion based on the combat motion feature and anevaluation function between the feature from the user’s queryand the feature from each pair of combat motion in thedatabase.
A. Feature deﬁnition
The combat motion feature contains the ID numbers of thekey poses of attacking and defending ﬁgures Fpa and Fpd ,respectively. It also contains the relative horizontal position Fxand Fz and relative horizontal orientation Fo of the attacking
ﬁgure relative to the defending ﬁgure for the key poses, as
shown in Fig. 4. The range of relative orientation Fo isbetween −180◦ and 180◦ , and becomes 0◦ when the twoﬁgures face the same orientation.
The feature values for pairs of combat motion in thedatabase are computed from the position and orientation of thepelvis in the key poses. First, the key poses are projected on
the ground to obtain the horizontal positions and orientationsof the two poses. Relative horizontal positions Fx and Fz andorientation Fo are then computed. ID numbers Fpa and Fpd
are speciﬁed for each pair of combat motion.
The feature values for user input are computed from the ID
numbers, positions and orientations of two AR markers in thesame manner. Because the size of ﬁgures is scaled down to1/16, the relative position is scaled up to 16.
B. Evaluation Function
The evaluation function computes the distance between thefeature from each pair of combat motion Fpa , Fpd , Fo , Fx , Fz′′and the feature from the user’s query Fpa, Fpd, Fo′ , Fx′ , Fz′ .
Because the components of the feature are in different units,we compute the weight sum of these components, as describedin the following.
The smaller the distance between two feature values of eachcomponent, the closer the two features.

Fig. 4. The poses, and the relative position and orientation of the keyposesof combat motion are used as feature.

1) Distance between pose ID numbers: Distance Vpa be′
tween two pose ID numbers Fpa and Fpa
is computed as{′0 (if Fpa = Fpa).
(1)Vpa =1 (otherwise)′Distance Vp a is also computed from Fpd and Fpd
in the samemanner.
Because we currently use a limited number of ﬁgures,and their poses are completely different, we simply evaluate
whether two poses are the same. To consider many ﬁgures that
contain similar poses, the similarity between poses should beevaluated using a method such as in [8]. This extension will
be considered in our future work.
2) Distance between orientations: Distance Vo between
two relative orientations Fo and Fo′ is computed as{Vo =|Fo′ − Fo |(if |Fo′ − Fo | < 180).
′360 − |Fo − Fo | (otherwise)(2)

The angular distance is computed to be between 0◦ and 180◦ .
3) Distance between relative positions: Distance Vc between two relative horizontal positions (Fx , Fz ) and (Fx′ , Fz′ )is computed as√
Vc = (Fx′ − Fx )2 + (Fz′ − Fz )2 .
(3)
4) Overall Distance: Distance V between two features iscomputed as the weighted sum of all components:
V = αVpa + αVpd + βVo + γVc ,(4)where α,β and γ are scaling parameters.
α, β and γ are determined from the ratio of the range offeature quantities of the database. The scaling parameters α,βand γ can be determined based on the distribution of featuresin the motion database. In our experiment, we used α = 150,
β = 1.25 and γ = 1.
VI. E XPERIMENTS AND D ISCUSSION
A. Design of Experiment
To evaluate the effectiveness of the proposed system, we
compared our interface with a conventional GUI-based interface. Four subjects who had no experience of computer

TABLE I
E XPERIMENTAL RESULTS .
InterfaceFig. 5. Five target pairs of combat motion used in the experiment.

animation participated in our experiment. We chose ﬁve targets
of combat motion from 60 pairs of motion in our database sothat they included different combinations of poses, relativepositions and orientations (Fig. 5). During the experiment, atarget pair of combat motion was presented and the subjects
were asked to search for target motion using our interface
and the GUI-based interface. We measured and evaluated therequired operation time and the number of trials until thesubject found each target motion for each interface.
Because the module for recognizing the AR marker was
not implemented, we used the Wizard of Oz method. A
human operator interpreted the subject’s query and input
the same query using the GUI-based interface. We manuallymeasured the time taken for the subject to select ﬁgures andarrange them on the platform as the operation time for ourexperiments. Therefore, there were small marginal errors in
the measurements.
B. GUI-based interface
We implemented a conventional GUI-based interface forcomparison. The user speciﬁed the same information used forour 3D ﬁgure interface using a mouse and keyboard. When theuser selected a ﬁgure on the screen, handles for manipulating
the horizontal position and orientation were displayed. The
user moved and rotated the ﬁgure by dragging the handles.
The types of ﬁgures were switched by pressing the associatedkey on the keyboard.
C. Experiment results
The average of the experimental results for the four subjects
is shown in Table I. In the experiment, because the ﬁgurebased interface was input manually using the Wizard of Oz
method, there was a slight measurement error in the result.
Overall, the operation time for our ﬁgure-based interface wasshorter than that of the GUI-based interface. The number oftrials for our ﬁgure based interface was also smaller thanthat of the GUI-based interface. The results indicate that ourmethod was effective in searching for the intended combat
motion from the database.

Figure-based

GUI-based

Taget motion

Time

No. of Trials

Time1

8.2

1

29.4

No. of Trials
12

5.5

1

29.6

1.5

3

8.0

1.5

30.1

24

11.7

1

7.3

15

6.3

1

12.2

1.5

average

7.95

1.1

21.73

1.4

Two subjects failed to ﬁnd the expected motion on theﬁrst trial with the ﬁgure-based interface for the target motion
3 (Fig. 5 (3)). This is probably because two poses wereplaced in diagonally opposed positions, as shown in Fig. 5(3), and the subjects were unable to reproduce the relative
position and orientation using the 3D ﬁgures. The four subjects
also required a longer operation time to use the ﬁgure-based
interface for the target motion 4 (Fig. 5 (4)). This is probably
because the initial arrangement of ﬁgures in the GUI-based
interface was close to the arrangement of the key poses in thetarget motion.
VII. C ONCLUSIONIn this paper, we proposed a tangible interface using 3D
printed ﬁgures to search for combat motion of two characters. Our experiments demonstrated the effectiveness of ourinterface. In future work, we will apply our system to a large
number of actual combat motion and conduct a user study.
Future work will also include the automatic detection of
key poses in pairs of combat motion and an extension of thedistance function to consider the similarity between poses.
VIII. ACKNOWLEDGMENT
This work was supported in part by a Grant-in-Aid forScientiﬁc Research (No. 15H02704) from the Japan Society
for the Promotion of Science (JSPS).
R EFERENCES[1] Yukawa Takashi, Obara Naoko, and Tamamoto Hideo. Construction ofthe motion database system whose contents can be searched by a motion
name. IEICE Technical Report, Vol. 104, No. 740, pp. 193–197, 2005.
[2] Tokuda Mio and Soga Asako. Retrieval system of motion data by humanpose input using a kinect. ITE technical report, Vol. 39.8, No. 0, pp.
73–76, 2015.
[3] Naoki Numaguchi, Atsushi Nakazawa, Takaaki Shiratori, and Jessica K.
Hodgins. A puppet interface for retrieval of motion capture data. In ACM
SIGGRAPH/Eurographics Symposium on Computer Animation 2011, pp.
157–166, 2011.
[4] Jungdam Won, Kyungho Lee, Jessica Hodgins, Carol O’Sullivan, andJehee Lee. Generating and ranking diverse multi-character interactions.
ACM Transactions on Graphics (SIGGRAPH ASIA 2014), Vol. 33, No. 6,p. 219, 2014.
[5] Jackie Assa, Yaron Caspi, and Daniel Cohen-Or. Action synopsis:
pose selection and illustration. ACM Transactions on Graphics (ACM
SIGGRAPH 2005), Vol. 24, No. 3, pp. 667–676, 2005.
[6] Optitracjapan. https://www.optitrack.co.jp/.
[7] Artoolkit. https://www.msoft.co.jp/ar/.
[8] Lucas Kovar, Michael Gleicher, and Frederic Pighin. Motion graphs.
ACM Transactions on Graphics (SIGGRAPH 2002), Vol. 21, No. 3, pp.
473–482, 2002.