情報処理学会 インタラクション 2016
IPSJ Interaction 2016

163C45
2016/3/4

Designing Interfaces for Generating Humanoid Hand Movements
By Sparsely Selecting Basis Trajectories
有木 由香1

Yuka Ariki1

稲邑 哲也2

Tetsunari Inamura2

池田 思朗3

Shiro Ikeda3

森本 淳4

Jun Morimoto4

Abstract: This paper proposes a robot interface design method by which we can control humanoid endeﬀector movements with such a low-dimensional input device as a gamepad. In our proposed method, first,
the numbers of movement trajectories to accomplish diﬀerent tasks are generated using a simulated robot
model and stored in a database. Second, a human user demonstrates the current task-related behavior.
Third, the corresponding stored movements for the demonstrated human behavior are sparsely extracted bya sparse coding method. Finally, the sparsely extracted movement bases are linearly combined to generate anovel movement to accomplish a new target task where the linear weight parameters are modulated by the
gamepad. We easily generated such complicated hand movements as spiral motions on a small humanoid
robot with our proposed interface.

1. INTRODUCTION

On the other hand, one of the most popular interfaces
to control high-dimensional systems is a gamepad. Us-

As robotic technologies continue to progress, robots areing such a low-dimensional input device as a gamepad to

expected to support daily human activities, engage incontrol high-dimensional systems, including in-game char-

manufacturing, and assist disaster-recovery eﬀorts. To

acters, is a very common interface implementation.

use robotic devices for supporting human activities, weInterestingly, users do not need to spend a significant

must properly design an interface that connects humanamount of time for training to achieve a certain skill level

users and robots. Using vision systems to monitor hu-

to properly control the in-game characters. This is prob-

man gestures for recognizing user intentions is a popu-

ably because the design that connects the input device tolar approach [1], [2], [3]. A hand movement recognition

the high-dimensional system is proper. The development

device such as a Wii controller has also been used as aof an proper robot interface, which allows users to control

robot interface [4]. Measuring myoelectric signals is an-

a complicated robotic device without significant eﬀort, isother promising approach to design an intuitive interface

critical.

to control robotic devices [5], [6], [7]. Brain machine inter-To develop such a useful robot interface, we need to findfaces are also getting attention as a potentially very usefulproper constraints to connect low-dimensional input de-

robot control interface [8], [9].

vices with a robot system. In previous studies, manifold

1

learning methods, e.g., [10], [11], have been used to find

2

3

4

Dept. of Science and Engineering, Ritsumeikan University
yayukaar@fc.ritsumei.ac.jp
National Institute of Informatics, The Graduate School for
Advanced Studies
inamura@nii.ac.jp
The Institute of Statistical Mathematics
shiro@ism.ac.jp
Dept. of Brain Robot Interface, ATR Computational Neuroscience Laboratories
xmorimo@atr.jp© 2016 Information Processing Society of Japan

low-dimensional representations of high-dimensional human movements as proper constraints [12], [13], [14], [15],[16], [17]. However, in these previous approaches, since
an appropriate low-dimensional manifold needs to be extracted for each target behavior, they are not suitable forgenerating a wide variety of movements.

947

情報処理学会 インタラクション 2016
IPSJ Interaction 2016

163C45
2016/3/4

図 1 Schematic diagram of our proposed framework. In our proposed method, 1) num-bers of movement trajectories to accomplish diﬀerent tasks are generated with asimulated robot model and 2) stored in a database; 3) human user demonstrates
current task-related behavior; 4), 5) corresponding stored movements and parameters to demonstrated behavior are sparsely extracted by a sparse coding method;6), 7) sparsely extracted movement bases are linearly combined to generate novel
movement to accomplish a new target task where linear weights for combinations
are controlled by gamepad.

On the other hand, task-space control is a standard

This paper is organized as follows. In Section 2, weapproach to generate high-dimensional whole-body hu-

explain our strategy for constructing a low-dimensional

manoid movements that correspond to task-relevant end-

interface for humanoid end-eﬀector control with a sparse

eﬀector trajectories [18], [19], [20]. However, it is not verycoding approach. In Section 3, we introduce our exper-

easy to generate three-dimensional task-space hand move-imental setups that are composed of a small humanoid

ments by a gamepad when users need to generate compli-

robot platform and a low-dimensional input device. In

cated hand trajectories. In addition, since users need to

Section 4, we show our experimental results using thecontinuously control a humanoid end-eﬀector using thesimulations and a real humanoid robot. We also present

input device, generating fast movements is also diﬃcult.

control performances with a designed humanoid interface,In this paper, we consider a diﬀerent approach for awhich we compare with a standard task-space control in-

robot interface design. We adopt the ideas of movement
primitives [21] or motor tapes [22] to generate robot movements. Previous studies showed that these approaches are

terface.

2. Methods

useful for generating a variety of movements from previ-

Here we introduce our interface design method for hu-

ously learned motions to adapt to the surrounding envi-

manoid end-eﬀector control. Figure 1 shows the inter-ronments and the given tasks [22], [23]. In our approach,face design strategy. First, we present how we store basis

we first store diﬀerent trajectories as basis movements

movements into a database by using a simulated environ-that can be considered movement primitives. Then wement. Second, we introduce how to extract basis move-use a sparse coding approach by which a newly observed

ment trajectories from the stored movements. Finally,

movement is represented with sparsely selected basis tra-we describe how to generate humanoid end-eﬀector move-jectories. By linearly combining the selected basis trajec-

ments by using a low-dimensional input device such as atories, users can generate diﬀerent types of movements,

gamepad based on the extracted bases.

where the combinations of the bases is determined using
a low-dimensional input device such as a gamepad.

© 2016 Information Processing Society of Japan948

情報処理学会 インタラクション 2016
IPSJ Interaction 2016

163C45
2016/3/4

2.1 Storing movement bases
We first store numbers of end-eﬀector movement trajectories in a movement database by using a simulated
humanoid robot model. It is easy to design wide variety
of movements for accomplish diﬀerent tasks in a simulated
environment. We then solve inverse kinematics problems
to generate corresponding joint movements when we control a humanoid platform. A simulated robot generates
numbers of task-related hand movements and these end-

図

2 (A) 25-DOF small humanoid platform,(B) Low-

eﬀector trajectories are stored as movement bases. Heredimensional input device. The device has two of two-DOF

we assume that we have N trajectory bases, where eachanalog joysticks.

end-eﬀector trajectory basis has m sample points. Then,the movement basis storage can be represented as:
D = [b1 , b2 , . . . , bN ] ,3. Experimental setups(1)

3.1 Small humanoid robot platform and Low-

where bi ∈ Rm denotes each trajectory basis.

dimensional input device
We use a 25-DOF small humanoid platform (see also

2.2 Extracting movement basesFig.

2(A)) to evaluate our proposed interface design

We then linearly combine end-eﬀector trajectory ba-

method. As an input device, we use a gamepad whichsis, where the trajectories are sparsely extracted from thehas two of two-DOF analog joysticks (see also Fig. 2(B)).

movement database. For selecting these basis, we consider

The analog joystick inputs were used to determine the pa-

using a sparse coding method based on L1 -norm regular-

rameter inputs ∆x in (4). Then, a trajectory composed of

ization [24]. Concretely, we solve a sparse linear regression

the linearly combined bases was used to control the small

problem:humanoid platform.

x∗ = argminx ||y − Dx||22 + λ||x||1 ,(2)

where x denotes weight parameter vector and y denotes anobserved hand-movement trajectory of a human demonstrator. Sparsity of the solution depends on the parameterλ. x∗ denotes optimized weight parameter.
2.3 Generating movements by constructed interface
Finally, we generate task-related end-eﬀector movements by controlling the weight parameters that correspond to the extracted basis trajectories. To control theweight parameters, we use low-dimensional input device

図 3

An example of the stored spiral trajectories that are usedin our experiment, and corresponding robot postures.

such as a gamepad. By using the modified weight parameters, end-eﬀector movement trajectories are representedas:

3.2 Constructing spiral movement databaseỹ = Dx̃,(3)

We constructed end-eﬀector movement database using
a simulation environment of the small humanoid platform

where

depicted in Fig. 2(A). In this study, we consider generat-x̃ = x∗ + ∆x

(4)denotes the modified input parameter vector composed of∗ing complicated hand movements such as spiral trajectories. In our experiment, 96 spiral trajectories are gener-

the parameter find in the reconstruction process x in (2)ated in the simulated environments, where the spirals withand the low-dimensional control input ∆x. ỹ denotes thefour diﬀerent sizes, three diﬀerent length, and eight dif-

newly generated robot movement trajectory.

ferent rotation angles were considered, i.e., 4 × 3 × 8 = 96.

© 2016 Information Processing Society of Japan949

情報処理学会 インタラクション 2016
IPSJ Interaction 2016

163C45
2016/3/4

Then, the generated trajectories were stored and repre-

for each diﬀerent demonstrated movement. By using thesented in a matrix from D in (1). Figure 3 shows anconstructed robot interface, we showed that the user was

example of the stored spiral trajectories that were used inable to easily control the end eﬀector of the small hu-

our experiment, and corresponding robot postures.

manoid robot by using the gamepad input device basedon the sparsely selected basis trajectories. Figure 5 shows

3.3 Spiral movement demonstration
From the constructed movement database, we sparsely
extracted four movement bases by observing a hand move-the real humanoid robot movement when the robot generated the modulated spiral movement which is presented
in Fig.4(C).

ment y demonstrated by a human user. As examples, ahuman user demonstrated three diﬀerent types of spiralmotions as depicted in Figs. 4(A),(D) and (G). Then, foreach observed spiral movement, corresponding trajectories
in the movement database are sparsely selected by the basis selection method presented in (2). The four movement
bases are selected based on the absolute value of the element of the optimized parameter vector x∗ in (2). Notethat we used lambda parameter λ = 0.001 so that lessthan ten movement bases were sparsely selected from thedatabase which includes 96 bases. In other words, based

図 5

Generated spiral movements on real humanoid platform.
Red disks represent end-eﬀector positions.

on the absolute value of the element of the optimized parameter vector, we finally chose the four bases from theless than ten bases selected by the sparse coding method.

4.2 Spiral movement control to generate a desired trajectory

3.4 Spiral movement generation

We then showed that the constructed humanoid inter-

The parameters for the selected basis trajectories areface were able to be used to follow a target spiral move-controlled to generate modified spiral movements. These

ment by controlling the small humanoid robot with usingparameters were controlled by using the gamepad (seethe gamepad input device. Although generating a spiralalso Fig.2(B)). The two of the two-DOF joysticks were as-

movement by using the constructed robot interface wassigned to control the selected four parameters. We show

easy, still, some training trials were necessary for a userthat the small humanoid platform can generate similar

to follow the target spirals. We show the learning perfor-

but diﬀerent spiral hand movements from the observed

mance of a user to generate a target spiral movement inspiral trajectories by controlling parameters with using

Fig. 6. Within 30 trials, humanoid end-eﬀector control

the gamepad. We then also show that we can control theperformance was much improved, where each trial only

small humanoid robot to generate the modulated spiraltakes around one second.

movement.

4. Results
4.1 Generating spiral movements

As a comparison, we also asked the same subject togenerate a target spiral movement by using an control
interface by which the hand position in a cartesian coordinate were directory controlled by the two-DOF analog

We first showed that we were able to generate spi-

joystick. Figure 7 shows the comparison of the generated

ral movements by using the low-dimensional input de-spiral movements. As in Figure 7(B), direct control of thevice. Figure 4 shows the results of the interface construc-hand position for generating the spiral trajectory by using

tion procedure. Human user demonstrated diﬀerent spi-

the gamepad was very diﬃcult while, as in Fig.7(A), it was

ral movements that have three diﬀerent shapes (see alsoeasy to generate spiral end-eﬀector movement to follow theFigs. 4 (A),(D) and (G)). For each demonstrated move-target trajectory by using the proposed interface. These

ment, the four movement bases were selected by using (2)results clearly show that our proposed approach was much(see also Figs. 4 (B),(E) and (H)) and the correspond-easier to be used to generate a complicated end-eﬀector

ing parameters were controlled by using the gamepad. In

trajectory such as a spiral movement.

Figs. 4(C),(F) and (I), modulated spirals are presented© 2016 Information Processing Society of Japan950

情報処理学会 インタラクション 2016
IPSJ Interaction 2016

図4

163C45
2016/3/4

Generated spiral movements. (A)-(C) correspond to first observed movement, (D)(F) correspond to second observed movement, (G)-(I) correspond to third observed
movement. (A),(D),(G): Demonstrated and reconstructed spirals. Demonstrated
movements were measured by using Visualeyez real-time motion capture system(Phoenix Technologies Inc.). (B),(E),(H): Selected basis end-eﬀector trajectories.
(C),(F),(I): Modulated spirals by using proposed interface.

5. CONCLUSIONSWe proposed a humanoid interface design method. We
showed that we can easily generate spiral hand movements
on a small humanoid platform by using the gamepad input device. Humanoid robot movements were controlled
by the combined basis motion trajectories, where the parameters to combine each basis were determined by the
図 6

Spiral control performance using the proposed interface.

input command specified with the gamepad. The basis

Within 30 trials, humanoid end-eﬀector control perfor-

trajectories were sparsely extracted from the stored spiralmance was much improved, where each trial only takes

trajectories by using the sparse coding method. Diﬀer-

around one second. Mean and standard deviation of er-

ent spiral movements were successfully generated by usingrors for every five trials are plotted.

the constructed interface. Since the stored basis trajectories were generated in the simulated humanoid robot system, it would not be always suitable to code the observed
human movements. In such case, we can possibly use a
dictionary learning method [25] to adapt the stored basis
motion to observed human movement trajectories. Therefore, in our future study, we consider using this kind of

図 7Comparison of the generated spiral movements. This re-adaptation mechanism to refine the stored basis trajecto-

sults clearly show that our proposed approach was muchries to properly represent observed human behaviors.

easier to be used to generate a complicated end-eﬀector
movement such as a spiral trajectory.

(A)Proposed.

(B)Control interface by which the hand position in a
Cartesian coordinate were directory controlled by the twoDOF analog joystick.

© 2016 Information Processing Society of JapanACKNOWLEDGMENT
This is the results carried out under MIC-SCOPE.
Part of this research was supported by SRPBS,
AMED/MEXT. This research was also partially sup-951

情報処理学会 インタラクション 2016
IPSJ Interaction 2016

163C45
2016/3/4

ported by MEXT KAKENHI grant Number 23120004,
ImPACT Program of Council for Science, Technology
and Innovation, and NEDO.
[15]

参考文献
[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

H. Do, C. Mouser, Y. Gu, W. Sheng, S. Honarvar, and
T. Chen, “An open platform telepresence robot with natural human interface,” in Cyber Technology in Automation, Control and Intelligent Systems (CYBER), 2013
IEEE 3rd Annual International Conference on, May
2013, pp. 81–86.
J. T. C. Tan and T. Inamura, “Sigverse - a cloud computing architecture simulation platform for social humanrobot interaction,” pp. 1310–1315, 2012.
R. Yan, K. P. Tee, Y. Chua, and H. Tang, “A gesturerecognition system using localist attractor networks forhuman-robot interaction,” in Robotics and Biomimetics (ROBIO), 2010 IEEE International Conference on,
Dec 2010, pp. 1217–1222.
D. Balakrishna, P. Sailaja, R. Rao, and B. Indurkhya,“A novel human robot interaction using the wiimote,”in Robotics and Biomimetics (ROBIO), 2010 IEEE International Conference on, Dec 2010, pp. 645–650.
T. Matsubara, S. Hyon, and J. Morimoto, “Learningand adaptation of a stylistic myoelectric interface: Emgbased robotic control with individual user diﬀerences,”in Robotics and Biomimetics (ROBIO), 2011 IEEE International Conference on, Dec 2011, pp. 390–395.
K. Kiguchi and Y. Hayashi, “An emg-based control for
an upper-limb power-assist exoskeleton robot,” Systems,
Man, and Cybernetics, Part B: Cybernetics, IEEE
Transactions on, vol. 42, no. 4, pp. 1064–1071, Aug 2012.
J. Furukawa, T. Noda, T. Teramae, and J. Morimoto,“An emg-driven weight support system with pneumatic
artificial muscles,” IEEE Systems Journal, vol. PP,
no. 99, pp. 1–9, 2014.
J. Zhao, Q. Meng, W. Li, M. Li, F. Sun, and G. Chen,“An openvibe-based brainwave control system for cerebot,” in Robotics and Biomimetics (ROBIO), 2013
IEEE International Conference on, Dec 2013, pp. 1169–
1174.
T. Noda, N. Sugimoto, J. Furukawa, M. Sato, S. Hyon,and J. Morimoto, “Brain-controlled exoskeleton robot forbmi rehabilitation,” in IEEE-RAS International Conference on Humanoid Robots, 2012, pp. 21–27.
P. Dollar, S. Belongie, and V. Rabaud, “Learning to traverse image manifolds,” in Proceedings of NIPS, 2006,
pp. 361–368.
P. Dollar, V. Rabaud, and S. Belongie, “Non-isometricmanifold learning: Analysis and an algorithm,” in Proceedings of the 24th international conference on Machine learning, 2007.
K. Tatani and Y. Nakamura, “Dimensionality reduction and reproduction with hierarchical nlpca neuralnetworks-extracting common space of multiple humanoid
motion patterns,” in Proceedings of IEEE International
Conference on Robotics and Automation, 2003, pp.
1927–1932.
K. Grochow, S. L. Martin, A. Hertzmann, and
Z. Popović, “Style-based inverse kinematics,” in ACM
SIGGRAPH 2004 Papers, ser. SIGGRAPH ’04, 2004,
pp. 522–531.
C.-S. Lee and A. Elgammal, “Human motion synthesis
by motion manifold learning and motion primitive seg-

© 2016 Information Processing Society of Japan

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

mentation,” in Proceedings of the 4th international conference on Articulated Motion and Deformable Objects,
ser. AMDO’06, 2006, pp. 464–473.
S. Bitzer, I. Havoutis, and S. Vijayakumar, “Synthesising novel movements through latent space modulation ofscalable control policies,” in SAB, 2008, pp. 199–209.
Y. Ariki, T. Inamura, and J. Morimoto, “Learning a humanoid robot interface by embedding a low-dimensionalcommand manifold into a high-dimensional joint action
space,” in Proceedings of IEEE/RAS the International
Conference on Humanoid Robots, 2013, pp. 514–519.
——, “Observing human movements to construct a humanoid interface,” in Proceedings of IEEE/RAS the International Conference on Humanoid Robots, 2014, pp.
342–347.
O. Khatib, “A unified approach for motion and force control of robot manipulators: The operational space formulation,” IEEE Journal of Robotics and Automation, vol.
RA-3, no. 1, pp. 43–53, 1987.
J. Nakanishi, R. Cory, M. Mistry, J. Peters, and
S. Schaal, “Operational space control: A theoretical andempirical comparison,” I. J. Robotic Res., vol. 27, no. 6,
pp. 737–757, 2008.
J. Peters and S. Schaal, “Learning to control in operational space,” I. J. Robotic Res., vol. 27, no. 2, pp.
197–212, 2008.
S. Schaal, “Is imitation learning the route to humanoid
robots?” Trends in cognitive sciences, no. 6, pp. 233–
242, 1999.
C. Atkeson and J. Morimoto, “Nonparametric representation of policies and value functions: A trajectory-based
approacha,” in Advances in Neural Information Processing Systems, vol. 15, 2002, pp. 1611–1618.
A. Ude, A. Gams, T. Asfour, and J. Morimoto, “Taskspecific generalization of discrete and periodic dynamic
movement primitives,” IEEE Transactions on Robotics,
vol. 26, no. 5, pp. 800–815, 2010.
R. Tibshirani, “Regression shrinkage and selectionvia thelasso,” J. Royal. Statist. Soc B, vol. 49, pp. 702–709,
1996.
J. Mairal, F. Bach, J. Ponce, and G. Sapiro, “Online
dictionary learning for sparse coding,” in Proceedings ofthe 26t International Conference on Machine Learning,
2009, pp. 689–696.

952