The 18th IEEE International Symposium on
Robot and Human Interactive Communication
Toyama, Japan, Sept. 27-Oct. 2, 2009"©2009 IEEE. Personal use of this material is permitted. However, permission toreprint/republish this material for advertising or promotional purposes or for
creating new collective works for resale or redistribution to servers or lists, or toreuse any copyrighted component of this work in other works must be obtained
from the IEEE."TuIAH.3

Vib-Touch: Virtual Active Touch Interface for Handheld Devices
Sho Tsuchiya, Masashi Konyo, Hiroshi Yamada, Takahiro Yamauchi, Shogo Okamoto and Satoshi Tadokoro
Virtual active touchwith a pointer

Abstract— Haptic interaction with handheld devices is limitedby space and size constraints that inhibit free hand exploration.
We developed a compact haptic interface called Vib-Touch,which is operated by fingertip via a pointing-stick input device
containing a tactile feedback. A cursor on the screen could
perform virtual exploration as a substitute for the finger movement. We call this technology Virtual Active Touch. We also
propose a tactile stimulation method to represent not only tactile
sensations, but the whole touch experience, including kinesthetic
senses and a sense of shapes perceived by a fingertip. This
study reports on the first prototype of the Vib-Touch interface
for handheld devices. We confirmed that the prototype could
provide friction sensation and geometric shape information
using the proposed friction display method.

Vibrator ona pointing stickFig. 1.

I. INTRODUCTION

the tactile stimulator is much smaller than a touch screen andprovides richer tactile sensation because a smaller stimulator
can more rapidly generate complex waveforms. We havealready confirmed the performance of Virtual Active Touchfor perceiving roughness [4] and friction [5].
The purpose of this paper is to propose a Virtual Active
Touch interface for handheld devices and to develop a
prototype. We induce human geometric shape perception
using the friction display as an example of its application.
First, we describe the concept of Virtual Active Touchand its applications and explain a friction display. Next,
we describe the handheld prototype of the Virtual Active
Touch interface. Finally, we examine human geometric shapeperception using the friction display.

Haptic interaction in handheld devices will increase theirusability and provide impressive new content that visual
information alone cannot provide. Several researchers havedeveloped handheld tactile displays [1], [2]. However, theirmain objective was not to provide rich tactile sensation, but
to provide haptic icons as symbolic information only.
Our natural haptic perception requires active hand movement to perceive tactile properties, such as texture, friction,
shapes, and softness. One critical problem regarding haptic
interaction in handheld devices is that the small size ofthe hardware limits free-hand exploration. One possible
solution is a touch screen with vibratory tactile feedback.
Indeed, such a screen has already been installed on several
commercial mobile phones. For example, Immersion Corp.
has developed tactile feedback technologies for touch screens[3]. However, a vibration-based touch screen is limited in its
ability to provide rich tactile sensation because the entire
screen vibrates.
We have proposed the concept of ’Virtual Active Touch’ toovercome the size and weight limitations of a haptic device[4]. This concept is implemented by applying a tactile feedback function to a pointing-stick-type interface, as illustrated
in Fig. 1. The pointing-stick is operated by fingertip andgenerates tactile feedback. The cursor on the screen can beused to perform virtual exploration as a substitute for thefinger without any actual hand movement. In this approach,II. THE CONCEPT OF VIRTUAL ACTIVE TOUCH
A. Overview
Virtual Active Touch enables users of mobile interfaces
to interact with an object on the screen via a cursor used
as a virtual fingertip. This study proposes the addition of atactile feedback mechanism in a pointing-stick input device
to enable virtual active touch. A pointing stick operates bysensing applied force and enables the cursor to be manipulated by the application of a certain amount of force in thedesired direction. We call this technology Vib-Touch: ‘Vib’
is an abbreviation for both ‘Virtual active’ and ‘Vibration’.
Fig. 1 shows an example of a tactile interface to help
clarify our concept. The white arrow represents the cursor on
the screen. A vibrator is mounted on the pointing-stick input
device of the mobile phone and tactile feedback is provided,
corresponding to the cursor operation performed using the

S. Tsuchiya, M. Konyo, H. Yamada, T. Yamauchi, S. Okamotoand S. Tadokoro are with the Graduate School of Information Sciences, Tohoku University, 6-6-01 Aramaki Aza Aoba, Aoba-ku, Sendai,
980-8579, Japan {tsuchiya, konyo, yamada, yamauchi,

okamoto, tadokoro}@rm.is.tohoku.ac.jp

978-1-4244-5081-7/09/$26.00 ©2009 IEEE

Concept of virtual active touch [4]12fingertip. Using our proposed method, the generation of avibration in response to virtual active touch can produce various texture sensations [8], [9]. We have already confirmed
the performance of Vib-Touch for perceiving roughness [4]and friction [5].

FrictionN

3D ShapeN

ReactivemovementN

Edge

B. Advantages
The advantages of the Vib-Touch interface are summarized
as follows.
(1) Compact hardware to produce rich tactile sensation:
The Vib-Touch interface can be small in size because thesize of the tactile display is sufficient enough to cover an
area corresponding to one fingertip on the pointing stick.
Pointing sticks are compatible with handheld devices andare commonly used.
The Vib-Touch interface has an advantage in that itprovides richer tactile sensation compared to that produced
by touch screen vibrations. In the case of touch screeninterfaces, the entire screen vibrates. It is therefore difficult toconvert the vibrations to complex waveforms due to limited
actuation responses. However, the Vib-Touch interface canarrange smaller actuators locally. This ability to arrange
smaller actuators is advantageous in generating complex
waveforms in order to produce rich tactile sensations.
In practice, we need to develop a compact vibrator that ismountable on the pointing stick and that generates vibrations
in a wide frequency range with sufficient amplitude. The
authors have proposed an ultrasonic vibrator that can bemounted on the joystick of a game controller [10]. Thisvibrator will hopefully to be used for the Vib-Touch interface,although the generated force is not sufficient enough to
produce rich tactile sensations at present.
(2) Adaptability for the screen size: The Vib-Touch interface has few restrictions in terms of the size of the stroking
area because cursor movement does not require actual hand
movement. For example, if a touch screen tactile display isused for small screens, it is impossible to stroke the object
in such a small area. This limited contact results in a lackof immersive sensation in the interface. However, the VibTouch interface can adjust the cursor movement according
to the screen size. In the case of a small screen, the cursor
movement can be restricted by reducing the force scaling
factor. With such an adjustment, it is expected that users ofmobile interfaces can touch small or large objects on thescreen and experience the textures of those scaled objects.
(3) Compatibility with visual effects: The Vib-Touch interface uses the cursor as a virtual finger; operators do not cover
the screen with their actual finger. This is compatible withadding visual effects for contact phenomena to the objects on
the screen. With touch screens, users cannot avoid obscuringvisual cues with their fingers.

(a) Support for GUI
Fig. 2.

(b) 3D contact(c) Haptic interaction

Applications of Vib-Touch interface

C. Applications
We have confirmed that the Vib-Touch interface enables
users to feel texture [4]. The ability to feel more complex
textures, such as that of a cloth, is expected to be producedby the tactile synthesis method proposed by the authors [8].
However, we need more useful applications in order to popularize haptic interaction in handheld devices. This study proposes a new approach to induce human kinesthetic sensation
using tactile stimulation. Kinesthetic sensation, also known
as proprioceptive sensation, is highly related to our motor
control and recognition of 3D environments. Conventional
haptic interfaces have produced kinesthetic sensation using
force feedback devices. However, force feedback devices are
too large and too heavy to mount on handheld devices. If
kinesthetic sensations can be represented by a small tactile
stimulator, we can develop useful applications on handheld
devices.
We have proposed a friction display method using vibratory stimulation instead of lateral force [9]. Vibratory
stimulation is generated in response to stick-slip transitions
between the skin and the contact surface area, dependingon hand exploration speed and normal force. We confirmedthat friction sensation was represented by tactile stimulation
without tangential force.
Fig. 2 shows our targeted applications. As shown in Fig.
2(a), the proposed friction display method can be used torepresent friction sensation of sliding objects and sensationof stroking the edge of virtual objects.
In addition, we expect that the cutaneous friction sensation
can reproduce a human perception of geometric surface
shape, as shown in Fig. 2(b). The perception of geometricshape is regarded as kinesthetic information [6]. RoblesDe-La-Torre and Hayward [7] determined that lateral force
between the skin and the contact surface area is the maincue in perceiving geometric shape through active touch. If
any tactile sensation can substitute for lateral force in shapeperception, we can present three-dimensional shapes on a
two-dimensional screen without force feedback devices. The

13

shape display can be applied when representing GUI icons
such as a button or a slider, 3D-maps, or the movement of
shapes.
The last example, as shown in Fig. 2(c), represents thereactive movement of the virtual object. We expect that the
Vib-Touch interface can produce the sensation of relative
movement between the virtual finger and the virtual object
on the screen by connecting with visual cues. This sensation
can be used for representing the reactive movement of virtual
characters.
In this study, we focus on verifying the 3D shape display
induced by the cutaneous friction sensation.

x(t)

k

m
N

F

∆µ

c
0

x(t)

V

Starting pointof stickFig. 3.

III. FRICTION DISPLAY BY VIBRATORY STIMULI

1-DOF vibration model of stick-slip motion

B. Control of Vibratory Stimuli

We use a vibrotactile display method that presents friction
sensations to users [9]. The method focuses on the stick-slip
contact of finger skin with an object. The method delivers
friction sensations by controlling the activity of FA II typetactile receptors at the moment of stick-to-slip transition
using high frequency vibrations greater than 200 Hz. These
are selected to fit the frequency response characteristics ofthe FA II type receptor. We used the same stick-slip friction
model in a previous paper and improved vibratory stimuli
waveforms to produce a more natural friction sensation.

Fig. 4 shows the approach adopted in this report tocontrol vibratory stimuli for simulating friction sensations.
The finger pad is given vibratory stimuli according to thetiming of the stick-to-slip transitions calculated using the
stick-slip friction model. We improved a waveform to havea gradual increase of vibratory amplitude at the momentof stick-to-slip transitions. The previous study used a stepincrease, which caused a ragged surface sensation.
The intensity of FA II type receptors’ activities is controlled by modulating the amplitude of the applied voltage.
The maximum amplitude A of the voltage applied to avibrator is expressed as

A. Stick-slip Friction Model
The stick-slip phenomena of the contact between finger
skin and object surface were simulated using a 1-DOF
vibration model with Coulomb friction, shown in Fig. 3.
The model approximates the dynamic characteristics of thesurface of a finger pad in contact with a flat object in theshear direction. The model involves stiffness k, mass m, andviscosity c. The finger contacts a flat object with normal force
N , and lateral friction force F . The velocity of the object
is V , and ẋ(t) is the relative velocity between the skin andthe object. The contact state of the finger (stick or slip) was
calculated based on this model. The details of the simulation
are described in the literature [9].
Stick-slip phenomena depend on the stiffness and thedifference ∆µ between the static friction coefficient µ s andthe kinetic friction coefficient µ k . For example, stick-slip
phenomena are less likely to occur with smaller ∆µ. In
order to observe the phenomenon with smaller ∆µ, the model
requires smaller V and higher N .
The parameters (k, m, c) in the friction model affectthe frequency of the stick-to-slip transition. We set theseparameters so that the frequency of the model was close
to that of a real finger pad in contact with a flat acrylic
board. This was measured by filming the contact with ahigh-speed camera. The selected parameter values were as
follows: k = 2.0 [N/mm], m = 1.0 × 10 −6 [kg], andc = 1.0 × 10−5 [Ns/mm].

A = B(µs N − cẋ),(1)

where B is constant. The amplitude is proportional to theelastic force at the moment of the stick-to-slip transition,and increases as the static friction coefficient µ s or thenormal force N in the friction model increases. The vibration
frequency is constant at 300 Hz. After switching from the
stick state to the slip state, the applied amplitude increases
linearly for 5 ms and then decreases for 30 ms in a quadratic
form.
Fig. 5 shows waves of applied voltage (normal force N =0.5 or 2.0). Both the amplitude of the applied voltage andthe period of the stick-to-slip transition are influenced by N .
IV. EXPERIMENTAL APPARATUS
In order to validate the concept of Virtual Active Touch, a
handheld device was developed. The device was named the
Vib-Touch interface.
Fig. 6 shows the developed Vib-Touch interface. It is abox, the dimensions and weight of which are 185.5×81×39
mm and 268.1 g. A liquid crystal display (CENTURYCplus
one LCD-4300J) with 800 × 480 pixels was installed in thecenter of the device. A pointing stick with a tactile stimulator
was located in the right portion of the device, assuming users
would operate it with their right thumb.

14

Contact shoe
Voice coil

Stick-slip transition
Slip

Stick

Stick

Slip Stick

Slip

A

Force sensor

Appliedvoltaget

Fig. 4.

Control of vibratory stimuli
Fig. 7.

Pointigng-stick interface with a tactile stimulator

Amplitude [V]105Amplitude voltage
Envelope curve

Vib-Touch interface
Computerforvisualinformation

0
−5
−10
0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

Amplitude [V]Force sensor
LCD
Voice coil

System diagram

Amplitude voltage
Envelope curve

The pointing stick is shown in Fig. 7. It consists of a voicecoil (AEC, TACTAID VBW32) as a tactile stimulator anda six-axial force sensor (NITTA, EFS-18M20A25-M10). A
contact shoe was attached on top of the voice coil to provide
an adequate area between the fingertip and the voice coil. The
shoe is circular, with a diameter of 18 mm.
As shown in Fig. 6, the user of the device holds it withboth hands and manipulates a cursor on the computer screenwith his right thumb. When the user applies tangential force
with his fingertip to the pointing device, the cursor moves
according to the applied force. Depending on the cursor
velocity and the applied force in the normal direction of thepointing stick, vibrotactile stimuli are presented to the user
on the basis of the algorithm described in Section III.
Fig. 8 shows the signal flow of the entire system. The
force exerted by the user was sensed by the force sensor andtransferred to the computer with a real-time operating system(OS). The sampling frequency of the force data was 5 kHz.
The real-time OS computed the tactile stimuli delivered tothe user. Also, the cursor movement was computed by thereal-time OS and sent to another computer to produce visual
information.
We employed a linear transformation between the applied
force f (t) and the cursor velocity v(t). The transformation
equation is expressed as

0
−5

0

Cursor
Generating
movement tactile stimuli

Fig. 8.

10

−10

Force

Visual infromation

Time [s]a) Normal force = 0.5 N

5

RT-OS

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

Time [s]b) Normal force = 2.0 N

Fig. 5. Waveforms of the vibratory stimulation corresponding to changes
in normal force

185.5 [mm]Pointing-stick
81 [mm]

Display

Fig. 6.

v(t) = αf (t),Vib-Touch interface design(2)

where α works as a gain. Taking account of the size of thescreen, this value was set to 80 [mm/Ns] so that users can

15

Peak-to-peak displacement [ôm]

30
25

N
20

HeightFriction15

Width(a) Perceived bump

10

Object

5
0
0

2

4

6
Voltage [V]810CursorFig. 9. Relationships between applied voltages and generated displacementson the driven frequency of 300 [Hz](b) Visual information on the screen

Normal load

easily manipulate the cursor. The velocity v(t) corresponds
to V in the friction model in Fig. 3.
Fig. 9 shows peak-to-peak displacements generated by thevoice coil when sinusoidal voltages of 300 Hz, used for theexperiment in this study, were applied. It showed a linear
relationship between the applied peak-to-peak voltages andgenerated displacements.

N2
k

ÍN
1
L

N1
L0

L1

L
L2

L3

L4 Position(c) Normal load function

V. EVALUATION OF THE PERCEPTION OF
SURFACE SHAPES BY FRICTION SENSATION

Fig. 10. Tactile height illusion via virtually controlling normal finger-force

A. Purpose
computer screen, as shown in Fig. 10. After the virtual
normal force was increased to N 2 , it was decreased fromN2 to N1 over a span L. The normal force was N 1 = 0.5
and N2 was on the middle of the screen. The normal force
N (x) at position x was expressed as
N1(L0 ≤ x < L1 ) k(x − L ) + N(L1 ≤ x < L2 )1
1
N (x) =(3)−k(x − L2 ) + N2 (L2 ≤ x < L3 ) N
(L3 ≤ x < L4 ),1

We examined the possibility of representing geometricshapes by changing the friction sensation as an example ofkinesthesia induction by tactile perception, as described in
Section II. We confirmed whether geometric shapes could beperceived by changing the friction during touch movement.
In this study, we aimed to represent the shapes of bumps.
To control the friction sensation, normal force was increased artificially. The method we suggested for synthesizing a friction sensation is based on the Coulomb friction
model. Therefore, as normal force increases, lateral force(fixing strength) increases and as normal force decreases,
lateral force (fixing strength) decreases. Moreover, the periodof the stick-slip transition is influenced.
It appears that a higher bump is perceived as the amountof normal force change, that is, ∆N is larger. A wider bumpis perceived as the span L of the change of normal force, as
shown in Fig. 10.
We confirmed whether the height and width of a bump
could be represented by changing the friction sensation andwe examined the effects of ∆N and L.

where

k = (N2 − N1 )/(L2 − L1 ).

(4)

Five values of ∆N = 2.0, 3.0, 4.0, 5.0, 6.0 [N] and a valueof L = 20 were used to evaluate the effect of ∆N on theperceived height of the bump. Five values of L= 10, 15, 20,
25, 30 [mm] and a value of ∆N = 4.0 were used to evaluate
the effect of L on the perceived width of the bump.
Since the aim of this study was to confirm that participants
could perceive 3D shapes only with tactile stimuli, visual
information relating to the 3D shapes was not presented tothem. The visual information given to the participants wasa blue square representing the cursor, which was approximately 3 × 3 mm, as shown in Fig. 6.

B. Task and Procedure
The virtual normal force was increased from N 1 to N2
over a span L when a user stroked a virtual object on a16

Maginitudes of perceived height

1.8
1.6
1.4
1.2
1
0.8
0.6
0.4
0.2
0

and width of the bump. We confirmed that the height wascontrolled by ∆N and the width was controlled by L.
VI. CONCLUSIONS

L=20 [mm]132Fig. 11.

Magnitude of perceived widthThis study proposed the concept of Virtual Active Touchand developed a prototype for the Vib-Touch interface. We
reported a new approach to induce human geometric shapeperception using the proposed friction display method. We
implemented the friction display method for the prototype
and confirmed that the proposed method could control theheight and width of bumps. Representing shapes using the
proposed method is expected to be applied when representing
things such as GUI icons such as a button or a slider, 3Dmaps, and the movement of shapes, etc.

45ÍN [N]

6

7

Magnitude of perceived height

1.8
1.6
1.4
1.2
1
0.8
0.6
0.4
0.2
0

VII. ACKNOWLEDGMENTS
This research was supported by the Ministry of Internal
Affairs and Communications, Grant for the Strategic Information and Communications R&D Promotion Programme(SCOPE) #082102006.
R EFERENCES[1] J. Luk, J. Pasquero, S. Little, K.E. MacLean, V. Levesque, and V.
Hayward, A Role for Haptics in Mobile Interaction: Initial Design
Using a Handheld Tactile Display Prototype. Proc. of the ACM 2006
Conf. on Human Factors in Computing Systems (CHI2006), pp.171180, 2006.
[2] Q. Wang and V. Hayward. Compact, portable, modular, highperformance, distributed tactile display device based on lateral skindeformation. In Proc. IEEE HAPTICS’ 06, pp.67-72, 2006.
[3] http://www.immersion.com/[4] T. Yamauchi, M. Konyo, S. Okamoto, and S. Tadokoro, Virtual Active
Touch: Perceived Roughness Through a Pointing-Stick-Type Tactile
Interface, The Third Joint Eurohaptics Conference and Symposium
on Haptic Interface for Virtual Environment and Teleoperator Systems(WorldHaptics 2009), pp. 605-610, 2009.
[5] S.Tsuchiya, M. Konyo, H. Yamada, T. Yamauchi, S. Okamoto, and
S. Tadokoro, Virtual Active Touch II: Vibrotactile Representation ofFriction and a New Approach to Surface Shape Display, IEEE/RSJ
IROS2009 (in submitting).
[6] L.A. Jones and S. J. Lederman, Human Hand Function, New York:
Oxford University Press, 2006.
[7] G. Robles-De-La-Torre and V. Hayward, Force Can Overcome Object
Geometry In The Perception Of Shape Through Active Touch. Nature,
Vol. 412, pp. 445-448, 2001.
[8] M. Konyo, A. Yoshida, S. Tadokoro, N. Saiwaki, A tactile synthesis
method using multiple frequency vibration for representing virtual
touch, Proc. of IEEE/RSJ IROS2005, pp.1121-1127, 2005.
[9] M. Konyo, H. Yamada, S. Okamoto and S. Tadokoro, Alternative Display of Friction Represented by Tactile Stimulation without Tangential
Force, Proc. of EUROHAPTICS 2008, pp.619-629, 2008.
[10] M. Konyo, Y. Motoki, H. Yamada, S. Tadokoro and T. Maeno, “Producing Distributed Vibration by a Single Piezoelectric Ceramics fora Small Tactile Stimulator”, Proc. IEEE/RSJ International Conferenceon Intelligent Robots and Systems, pp.3698-3704, 2008.

ÍN=4.0 [N]5

10Fig. 12.

15

20 25
L [mm]

30

35

Magnitude of perceived width

A magnitude estimation method was used to evaluate
the perceived height and width of the bump. Participants
were asked to subjectively assign an arbitrary number toquantify the perceived height and width within 30 seconds.
Each stimulation combination was presented twice. Eachparticipant evaluated one set (20 trials) of experiments about
perceived height and width. The order of the stimulation
combinations was random and the order of the experiments
was changed. Participants practiced before the experiments.
They wore headphones through which pink noise was played
to cover the sounds generated by the vibrator. Five men and
one woman aged from 20s to 30s performed the tasks.
C. Experimental Result
The magnitudes of perceived height and width were calculated as the geometric average of the normalized resultsfrom all participants. Fig. 11 shows that the perceived heightincreased as ∆N increased. Fig. 12 shows that the perceived
width increased as L increased. These results corresponded
to our expectations.
The proposed method was used to represent the height17