Session: LBR Highlights

March 5–8, 2012, Boston, Massachusetts, USA

Development of a Jenga Game Manipulator
Having Multi-articulated Fingers
Tsuneo Yoshikawa

Tatsuya Sugiura

Seiji Sugiyama

Ritsumeikan UniversityKusatsu, Shiga 525-8577
Japan

Ritsumeikan University
Kusatsu, Shiga 525-8577
Japan

Ritsumeikan University
Kusatsu, Shiga 525-8577
Japan

qwer.all.releave.140691ru.ta28@gmail.com

seijisan@is.ritsumei.ac.jp

yoshikawa@ci.ritsumei.ac.jp
ABSTRACT

2.

This paper describes current status of our eﬀort to developa robot system that can play Jenga game against humanplayers. Unlike most of the previous Jenga robots that usegrippers, this robot is equipped with a hand with two multiarticulated ﬁngers covered by soft skin and does not haveany major mechanical constraint in playing the game in anatural way as human players do. An experimental resultof a game played between the robot and a human player ispresented.

Categories and Subject Descriptors: I.2.9 [Robotics]:
Manipulators

General Terms: Experimentation
Keywords: robot, manipulator, Jenga, multiﬁngered hand
1.

TASKS NEEDED IN PLAYING JENGAThe tasks needed in playing the game can be divided into
the following ﬁve basic tasks.
1. Recognition of the shape of the tower.
2. Selection of the block to be removed.
3. Performing the motion of removing.
4. Selection of the position for putting the removed blockon the top level.
5. Performing the motion of placing the removed block back.
Tasks 2 and 4 correspond to the strategy and tasks 3 and5 correspond to the skill of the game. Our approach for
performing each of these tasks is explained below.

2.1 Recognition of Block Positions
The block positions in the tower is recognized using theimages taken by a monocular camera attached to the baseof the hand (camera in hand).

INTRODUCTION

2.2 Selection of Block for Removal

Recently various game playing robots have been developed. Among them, there are several works on robots thatplay Jenga [1] [2] [3]. They all use either grippers that are
custom-made for Jenga or simple grippers available on themarket.
In this paper, we use a robot manipulator having a versatile hand with two multiarticulated ﬁngers which we havebeen using for research of advancing grasping and manipulation skill of multiﬁngered hands [4] [5] . Note that we havereported on a preliminary study of this system in a previous
paper [6]. Since then several major improvements have been
implemented: (1) Time needed for visual information processing was made shorter by replacing the omnidirectional
camera by a simple monocular camera and by reducing thenumber of necessary images, (2) ordinary Jenga blocks commercially available are now being used for experiment due
the the thinner ﬁngertips we are using now (in [6] blocks
made of two regular blocks bonded together were used, and(3) the ﬁngers grasp a block from the side instead of grasping it from the top, making it possible to play with higher
Jenga tower (In [6] playing with only three level towers werepossible). Results of these improvements are presented inthis paper.

Search for candidate blocks for removal is done based on
the consideration that the tower should not fall after removing the block. The basic idea here is that a suﬃcient condition for the stability of the tower is that the vertical linegoing through the center of gravity of the set of all blocks
from the top level to a certain level k (counted from thebottom) intersects with the support region deﬁned by thesmallest convex set including all the upper surfaces of theblocks in level k − 1. If there are multiple candidate blocks,
then we select one block based on a evaluation function representing the degree of danger. Currently the safest block is
selected according to this degree of danger, although there
can be other strategies that do not select the safest one.

2.3 Selection of Place for Putting Block
Selection of the place for putting back the removed blockon the top level is also done in a similar way to that ofremoval block selection.

2.4 Removing Motion
One diﬃculty in playing Jenga game is that the size ofthe blocks slightly diﬀer to one another. Because of this thediﬃculty of removing a block varies even if the geometricalcondition of the block in the tower is the same. To copewith this problem, the following scheme is adopted. After
grasping the block, the angle of the grasped block is slightlytilted. By this tilting, it is expected that the the contact
point will move to the outside edge of the block and theCopyright is held by the author/owner(s).
HRI’12, March 5–8, 2012, Boston, Massachusetts, USA.
ACM 978-1-4503-1063-5/12/03.

277

Session: LBR Highlights

March 5–8, 2012, Boston, Massachusetts, USA(a) t=0[s](b) t=4[s](c) t=18[s](d) t=28[s](e) t=60[s](f) t=80[s](g) t=94[s](h) t=157[s](i) t=183[s](j) t=192[s](k) t=240[s](l) t=285[s](m) t=339[s](n) t=341[s](o) t=351[s]Figure 1: Experiment of match-up

distribution of the load among the blocks on the level ofremoval block will change, hopefully, in such a way that theload of the block of removal will decrease.

We believe that a robot system that has a hand similar tohumans will be important for playing Jenga with humanplayers.

2.5 Putting Motion

5. REFERENCES

When a human player performs the putting motion of ablock on the top of the tower, it is done so that any large
impact force may not be applied to the tower. One way forachieving this by the robot is to detect the contact between
the block and the top surface of the top level of the towerby using the force sensors at the ﬁngertips.

3.

[1] T. Kröger, B. Finkemeyer, S. Winkelbach, L.-O. Eble,
S. Molkenstruck and F. M. Wahl, “A Manipulator
Plays Jenga,” IEEE Robotics and Automation
Magazine, vol.l5, no.3, pp.79-84, 2008.
[2] J. Wang, P. Rogers, L. Parker, D. Brooks, M. Stilman,“Robot Jenga: Autonomous and Strategic Block
Extraction,” In Proc. of The 2009 IEEE/RSJ Int.
Conf. on Intelligent Robots and Systems,pp.5248-5253, 2009.
[3] S. Kimura, T. Watanabe, Y. Aiyama, “Force Based
Manipulation of Jenga Blocks,” In Proc. of The 2010
IEEE/RSJ Int. Conf. on Intelligent Robots andSystems, pp.4287-4292, 2010.
[4] Tsuneo Yoshikawa, “Multiﬁngered Robot Hands:
Control for Grasping and Manipulation,” Annual
Reviews in Control, vol.34, issue 2, pp.199-208, 2010.
[5] T. Yoshikawa, M. Koeda, and H. Fujimoto, “ShapeRecognition and Optimal Grasping of Unknown
Objects by Soft-Fingered Robotic Hands withCamera,” Experimental Robotics: The 11thInternational Symposium, Springer Tracts in
Advanced Robotics, pp. 537-546, Vol. 54, 2009.
[6] T. Yoshikawa, H. Shinoda, S. Sugiyama, and M.
Koeda, “Jenga Game by a Manipulator with
Multiarticulated Fingers,” Proc. AIM2011, pp.960-965,
2011.

EXPERIMENT OF MATCH-UP

Experimental match-ups between the robot and a humanplayer was performed using ten level tower consisting of 30
blocks. Fig.1 shows a case in which the robot performed
very well. Fig.1(a) was the initial state. The human player
took the turn ﬁrst. The human player removed a block fromlevel 5 (Fig.1(b)) and put it back to the top (Fig.1(c)). Thenthe human player signaled the completion of his turn. Notethat the game proceeds automatically except for a signallingby the human player of the completion of his motion byclicking a key of a PC controller of the robot. Receiving thissignal, the robot started to take two images of the tower
from two neighbouring sides (Fig.1(c) and (d)) to estimate
the positions of the blocks in the tower by using the camera.
After repeating this process several times the robot, as its
fourth move, tried to remove a block from level 2 (Fig.1(m)and (n)) but the tower collapsed (Fig.1(o)).

4.

CONCLUSIONS

Although the present level of the robot system in performing the game is still very low, A variety of ideas could
be tried in this system to improve the skill of the system.

278