WISS2007

コインサイズのワイヤレスセンサを用いた Hands-Popie
Hands Popie using coin size wireless sensor

Ayman Atia

Daisuke Sato

Shin Takahashi

Jiro Tanaka ∗Summary.
Human gestures are typical examples of non-verbal communication, and help people
communicate smoothly [1]. However, using camera to recognizing gesture needs high processing powerand suﬀer from delays in recognition [2]. Popie [3] is a pen-based menu interface that facilitate theentering of Japanese text, but the user must be attached to screen in order to use this interface. So ourmain motivation is how we should design a user interface that use cookie wireless sensor [4] as an input
device. In this paper we describe the interface setting, method of recognizing full hand gestures anddirection from 3D-accelerometer data.

1

BackgroundPopie a menu-selection-based Japanese input
method for a pen device. The system starts by
allowing the user to select some characters constants then the system recommend the user a setof recommended words ”Kanji” to select from.
In some situations, consonant sequences can recommend a huge amount of kanji-form candidates.
The user can input consonants, and select kanjiform candidates ﬂuidly, using FlowMenu. [3].
In this research we attach a coin size wirelesssensor to cone or sphere object [5] and in our
primary experiments we put an assumption thatthe user not change the orientation of the sensor.
If the user change the orientation of the nokia
sensor then this will cause the gravity value ofthe accelerometer to be applied on the 3-axis accelerometer which at the end will cause a disruption for the accelerometer real values.
Nokia wireless coin size SensorsNokia cookie is an experimental device developed to test ubiquitous context aware applications. The chassis contains 6 sensors and sends
data using Bluetooth. The device is about thesize of 5 stacked quarter coins [4].
The sensor is composed of 2-axis linear accelerometer, Compass 3-axis sensor, Ambient lightsensor, Galvanic skin response sensor, Heart rate
sensor and Skin temperature sensor (see Fig.1and Fig. 2). The sensor has two communication∗Copyright is held by the author(s).
Ayman Atia, Daisuke Sato, Shin Takahashi and JiroTanaka, 筑波大学大学院 システム情報工学研究科 コン
ピュータサイエンス専攻

図 1. Nokia Sensor with 3D accelerometer(left)
and Nokia Sensor Extension boards(right)

interfaces, Bluetooth and UART wired connection [4].
In order to allow users to deﬁne there own gestures we use a gesture recognition toolkit called
GART [6] which depends on hidden markov model(HMM) for pattern recognition. GART has beentested using a regular mouse device to recognize
users gestures.

2Interface Overview

Popie composed of 24 basic patterns for example moving to north direction-northwest thenreturning back to initial state. We ask the user
to deﬁne the 24 basic patterns for Popie usage
see ﬁgure 2. Popie depends on basic 8 directions then each direction has 2 other directions
one to the east and other to west that can be
possible to moves towards. When reaching theinitial state this means that the user has ended
his pattern of motion and he will select one of therecommended words or enter another pattern ofmotion.
User has to train the Interface ﬁrst by using

WISS 2007

needs more enhancements in means of accuracy
and minimizing the eﬀect of gravity values thatare applied on the 3-axis of accelerometer whilechanging the orientation of the sensor.

5

図 2. Pattern for selecting North, North east
then back to start position.

the GART toolkit to deﬁne his own gestures foreach Popie pattern, user has a complete freedom
to determine his own hand gesture for each Popiepattern. For each pattern the user is asked togive at least ﬁve training sets. As much as theuser trains the interface the more accuracy for
detection the appropriate Popie pattern we canget.

3Interface ArchitectureIn interface split the patterns by using the(Galvinic skin reaction) sensor so that whenever
the user touch or release the GSR sensor the interface starts to record the pattern data. Theinterface captures and records a 3-d vector of theaccelerometer readings for the three axis (x,y,z)
respectively and calculates the angle between two
successive 3-d vectors and record this as one sequence of pattern. Hence a pattern composed ofthree readings of the 3d accelerometer and angle between vectors. The interface passes thisrecorded data to GART to perform the trainingof the patterns, after this the interface will beready to recognize the user s gestures.
When the user performs a certain gesture theinterface recognize the pattern and trigger thisinto a keyboard stroke. Popie application canbe operated by pen with touch screen or by keyboard strokes, so it is listening to keyboard strokes
and each keyboard stroke sequence can represent
a pattern.

4Primary evaluation

The primary evaluation shows that this interface can be useful to allow the user to freely
customize his gestures and interact with Popiewith his customized method of interaction. Onthe other hand the recognition of patterns stillConclusion

We developed an interface that allows usingof Nokia sensors wireless device as a tool for entering Japanese words and sentences using thePopie interface. Users can customize there own
gestures for each Popie pattern. so this can support the customization of the interface.
More study needed to evaluate this system inmeans of accuracy and time to enter text andcompare the results to the usage of pen-based
Popie interface. Also recognition accuracy could
be enhanced during time of usage.

参考文献[1] Tsukada, K. and Yasumura, M. “Ubi-Finger:Gesture Input Device for Mobile Use,” In Proceedings of APCHI 2002, Vol. 1, pp.388–400,(2002).
[2] Badillo et al. “Literature Survey on Interaction
Techniques for Large Displays,” TR-06-21, Computer Science, Virginia Tech (2006).
[3] D. Sato, B. Shizuki, M. Miura and J. Tanaka.
“Menu-selection-based Japanese Input Methodwith Consonants for Pen-based Computers,” In
Proceedings of APCHI 2004, pp.399–408, (2004).
[4] O. Hideki. “Nokia Cookie users manual,”Nokia
research center. Nokia Japan Co.,Ltd., (2005).
[5] A. Atia, S. Takahashi, and J. Tanaka. “Coin Size
Wireless Sensor Interface for Interaction with Remote Displays,” HCII2007, pp.733–742, (2007).
[6] K.Lyons,H.Brashear,T.Westeyn,
J.Soo,T.Starner. “GART: The Gesture and Activity
Recognition Toolkit,” HCII2007, pp.718–727,(2007).