Virtual Object Manipulation System with Substitutive Display of
Tangential Force and Slip by Control of Vibrotactile Phantom Sensation
Tatsuya Ooka and Kinya Fujita
Tokyo University of Agriculture and Technology

ABSTRACT
We propose a substitutive multi-degree-of-freedom force display
method that utilizes vibrotactile phantom sensation (VPS), in
which the tangential force is substituted by the VPS position
displacement and the slip is displayed as the oscillatory
displacement of the VPS position. Based on the proposed method,
we prototyped a virtual object manipulation system that consistsof two 20 mm x 20 mm x 20 mm fingertip-wearable devices withfour vibrating pins, a magnetic tracking system, a potentiometerfor finger position measurement, and a physics simulator. The
preliminary experiments demonstrated the feasibility to display
the mass of an object, the slipping-down of an object between
fingers, and the reaction torque induced by the rotation of theobject-grasping hand.
KEYWORDS: vibrotactile display, phantom sensation, tangential
force, slip
INDEX TERMS: H.5.2 [Information Interfaces and Presentation]:
User Interfaces— Haptic I/O
1

INTRODUCTION

Manipulation of an object is a task that is highly dependent on
haptic information. In order to attain precise object handling in a
virtual environment, the normal and the tangential forces as wellas the slip between the finger and the object are required.
Grounded fingertip force display devices, whether manipulatortype or string-type such as SPIDER [1], provide the multi-degreeof-freedom force display function. However, they have a more
complicated structure and more limited range of motion than otheroptions. Several types of pseudo-force display devices have been
proposed for a more simple structure and greater range of motion.
Fingertip tightening devices [2][3] are typical examples that weremade smaller by limiting the display information within thenormal force at the fingertips. In compensation for a simple
structure, the pseudo-force display devices, including the devices
based on other principles, lack the functions to display tangential
force and slip.
On the other hand, vibrotactile stimulation have been studied toassist haptic perception of contact state change [4][5]. Fingertip
vibrotactile display devices have been developed such as texture
display by using a pin array [6], slip display by using anelectrostatic actuator [7] as well as Braille display. 3D direction
display through the palm has also been attempted by controllingthe stimulus position using multiple vibrotactile stimulators [8].
Furthermore, vibrotactile phantom sensation (VPS) and apparent
movement, both of which allow stimulus position control with alimited numbers of stimulators have been applied to induce asensation of rubbing at the palm [9] and so on. The conventional
studies have utilized the vibration as a simple event cue or anavigational cue. Much of the haptic information duringcontinuous physical interaction has not been displayed. Therefore,
demand has arisen for a device that can display multi-degree-offreedom force and slip that allows users to feel the weight and slipof a grasped object.
In this paper, we propose a substitutive display method for
multi-degree-of-freedom force and slip that utilizes VPS. The
dynamic changes of normal force, tangential force, and slip
applied to the fingertip are substituted by controlling themagnitude and the position of the VPS. We also report on a multifingered virtual object manipulating system based on the proposed
method and the results of the virtual object manipulating tasks
which require the perception of tangential force and slip.
2

CONCEPT OF VIBROTACTILE SUBSTITUTION

The concept of the VPS substitution is shown in Fig. 1(a). The
delicate change of contact force, while holding an object with
fingers, is haptically perceived mainly by slow adaptive receptors
such as Merkel disks. Fast adaptive receptors, such as Pacinian
corpuscle, contribute to the perception of drastic contact statechange such as contact or release [10]. If the magnitude and theposition of the high-frequency VPS are dynamically controlled,
the user is considered to perceive the change of the VPS throughonly fast adaptive receptors, even if the changing frequency is low.
Therefore, although the contributing mechanoreceptors are limited
and different from the ones for static force perception, the2-24-16 Nakacho, Koganei, Tokyo 184-8588 Japan{50008646107@st, kfujita@cc}.tuat.ac.jp
IEEE Haptics Symposium 201025 - 26 March, Waltham, Massachusetts, USA
978-1-4244-6820-1/10/$26.00 ©2010 IEEE

Figure 1. (a) Conceptual diagram of the vibrotactile substitution of
the change of contact force. Proposed substitution methodsof (b) normal force, (c) tangential force and (d) slip.

215

adequate display of VPS and visual information in response tovirtual environment is expected to assist the haptic perception of
the contact state change..
The proposed idea is to substitute the forces and slip bycontrolling the applied vibration to the palmar surface of thefingertip as shown in Fig. 1(b, c, d). The normal force issubstituted by the vibration magnitude, similar to the previous
studies [3][4], that is most intuitive, as shown in Fig. 1(b). On theother hand, the substitution of the tangential force with normaldirection vibration is considered to be more difficult. We focusedon the fact that the tangential force generates the shear
deformation of the finger pad and generates the small
displacement almost proportional to the force. Therefore, we
assumed that the tangential force can be alternatively displayed bycontrolling the position of the VPS as shown in Fig. 1(c).
Furthermore, displaying the slip between the fingertip and themanipulating object is important in addition to displaying thenormal and tangential force to support haptic interaction between
users and the virtual environment. So-called slip is a stick-slip
micro vibration between the palmar surface of the fingertip andmanipulating object. Therefore, we hypothesized that the slip isalternatively displayed by the oscillatory modulation of the VPS
position that represents the tangential force as shown in Fig. 1(d).
3PROTOTYPE DEVICE AND OBJECT MANIPULATION SYSTEM

3.1
Prototype Device
The prototyped fingertip-wearable device is shown in Fig. 2. The
prototyped device has four pins for inducing two-dimensional
VPS at the palmar surface of the fingertip. Each pin was attached
to a vibrating coil (700 rolls，4 mm dia. x 8 mm) that is locatedaround a neodymium magnet (4430 G，3 mm dia. x 8 mm) and
was driven through 0.12 mm thick rubber film. The distance
between the pins was experimentally determined at 8 mm, themaximum distance for fused phantom sensation. The maximum
stroke of the free-vibrating pin, which is driven by the maximum
driving voltage, is 0.24 mm. The weight of one device without
cables is 14 g.
The four vibrations need to be recognized clearly as a fused
single vibration. At the same time, the vibration frequency isrequired to be sufficiently high as the carrier wave for theamplitude modulation to display slip. Therefore, the driving
frequency and the modulation frequency were experimentally set
at 200 Hz and 50 Hz, respectively. The modulation depth wascontrolled in proportion to the friction coefficients as well as theslip velocity.

Figure 3. Block diagram and configuration of the multi-fingered
virtual object manipulation system.

3.2
Virtual Object Manipulation System
After examining the basic property of the proposed substitutive
display method, a two-fingered virtual object manipulation system
was prototyped. The block diagram and the configuration of thesystem are shown in Fig. 3. The hand position of the user isdetected by a six-degree-of-freedom magnetic sensor (FOB,
Ascension) attached at the back of the hand, and the distance
between the thumb and the index finger is detected by apotentiometer. Based on these data, the positions of the thumb andthe index finger are calculated. In the virtual environment, which
is simulated by the physical simulator, two boxes were set as theproxies of the two fingertips. The two proxies allow users tomanipulate the virtual objects. The virtual environment issimulated using a general-purpose physics simulation library ODE.
Because ODE is based on an analytical method and requires
manipulation force as input, the virtual coupling (VC) thatconsists of elasticity and viscosity is utilized. The manipulation
force for simulation and the displayed force are calculated fromthe distance between the real finger and the proxy using VC. The
VC parameters were experimentally adjusted for distinct contact
recognition and simulation stability (elastic coefficient k = 800
N/m, viscous coefficient b = 30 N/m).
4

Figure 2. Structure of the prototyped fingertip-wearable VPS
display device.

216

EXPERIMENTS

4.1
Basic property of vibrotactile substitution
In order to verify the possibility of displaying the contact statechange by controlling VPS, the basic properties of the proposed
method were experimentally examined. At first, the correlation
between the VPS magnitude and the position and strength of theperceptive sense of normal force and tangential force wereexperimentally examined in order to investigate the performance
to display the contiguous change of contact force. Seven subjects
were required to voluntarily reproduce the perceived normal ortangential force by pushing the force sensor after the VPS
stimulation was applied at their palmar surface of the index
fingertip. The linear correlation for each subject (averagecorrelation coefficient 0.72 ± 0.12) was confirmed as expectedwith some degree of individual deviation.
Second, to investigate the possibility of object handlingassistance through the substitutive slip display, the complex

reaction time was evaluated by using a slip discrimination task
from simple increase of tangential force. The two types of VPS
stimulations were applied in random order synchronized withcomputer graphics (CG), which the touching virtual object slips or
moves minutely in the tangential direction. The subjects wererequired to push the force sensor as quickly as possible only ifthey felt the slip. In the same way, the reaction for real slip orincreased tangential force was examined by using a linear actuator.
Ten subjects performed the tasks under the following fiveconditions: VPS slip with/without CG, real slip with/without CG,
CG alone. The correct reaction rate of VPS slip without CG was
90%, and the average reaction time was 550 ms. The reaction time
was shortened to 220ms in the tasks without the necessity of
discrimination of slip and tangential force. The possibility torecognize and react to the VPS slip has been demonstrated.
4.2
Multi-fingered manipulation tasks
The multi-fingered multi-degree-of-freedom fingertip force
display system is expected to be able to assist tasks that require
the recognition of a complex state change. Therefore, thefollowing two virtual object manipulating tasks were performed.
4.2.1
Slip Down of Grasped Object
The task is to grasp and lift a virtual object and then let it slip
down by decreasing the grasping force. The subject is expected toperceive the normal force as the grasp reaction force at first, thetangential force induced by the mass of the object after lifting up,
and slip of the object at last. The mass of the manipulated virtual
object was 2.0 kg, the size was 7 cm x 7 cm x 20 cm and thefriction coefficient between the proxy and object was 0.85.
A typical example of the displayed force during the task is

Figure 4. Example of slip down of grasping object task. (a, b)
Displayed forces for thumb and index fingers, (c) relative
velocity between object and finger.

shown in Fig. 4. The y-axis forces (normal force) for the thumband index finger at the beginning are the grasp reaction force. The
x-axis forces (tangential force) are induced by the mass of themanipulating object. The decrease of the grasping force reduced
the friction force and the object starts slip at about t = 1.6 s as seenin Fig. 4(c). The oscillatory modulated tangential forces,
depending on the relative velocity and friction constant between
the proxy and manipulated object, are observed in Fig. 4(a, b).
4.2.2
Door Knob Turning
The task is to grasp a virtual door knob and turn it while therange-of-motion of the knob is limited. The torque caused by theknob rotation is expected to be perceived by the spatially opposite
tangential forces at the thumb and the index finger. The virtual
knob is a 9 cm x 9 cm x 9 cm cube. The rotation axis of the knob
is fixed and the rotation angle is limited up to ±15 degrees. The
minimal rotation torque of the knob was set at 0.1 Nm to allow
users to perceive the reactive torque against the rotation.
The typical result is shown in Fig. 5. The grasping of the knob
increased the normal force at the thumb and index finger as seenin Fig. 5(b, c). The following hand rotation, as seen in Fig. 5(a),increased the tangential forces in the same direction for both
fingers (they are opposite in the spatial coordinate system). Afterthe knob locking at about t = 6 s, the more rapid increase of thetangential force was observed that represents the increase of thereactive torque.
4.3
Subjective Evaluation
Six subjects were required to answer whether the state changes
during the two tasks can be haptically recognized or not, such as
the grasp reaction force, mass of the object, beginning of the slip,
the torque from the object, and the locking of the object.

Figure 5. Example of door-knob-turning task. (a) Rotation angle ofdoor knob and hand, (b, c) displayed forces for thumb andindex fingers.

217

Furthermore, the subjects who recognized each state change wererequired to evaluate the clarity of it using a five-level scale.
Fig. 6(a) shows the event recognition rates in the slip down task.
The event recognition rates of the mass and the slip were 100%.
However, the clarity scores of the slip were relatively low,
especially for the beginning of the slip. Fig. 6(b) shows the resultsof the knob turning task. The event recognition rate of the rapid
torque increase after the lock was 100%. However, event
recognition rates for the more mild torque increase during knob
rotation and onset of the knob lock were relatively low. The
clarity score of the door knob lock was lower than the other statechanges.
5

DISCUSSION

In the experiments to examine the basic properties of the proposed
substitutive force display method, the linear correlation between
the VPS position and the tangential force sense was verified. The
correct reaction rate of VPS slip without CG was 90%. The
possibility of the substitutive VPS display of the contact force andslip has been demonstrated. One point to keep in mind is that thereaction time for VPS slip without CG was 200 ms slower than
the real slip. The possible reason is that both tangential force andslip are substituted by the vibration while the real tangential forcechange is gradual and mainly perceived by the slow adaptive
receptors.
In the slip down task, all the subjects perceived the mass andthe slip. However, the clarity score for the beginning of the slip
was low. The most possible reason is that the oscillatory
modulation depth was insufficient because it was controlled inproportion to the slip velocity and the friction coefficient. In termsof event recognition assistance of contact state change from stablegrasp to slip, other rules that allow more deep modulation need tobe discussed.
In the knob turning task, all the subjects successfully felt the
opposite tangential forces to the thumb and index finger as torqueto the hand. However, some subjects failed to recognize the torqueincrease caused by the knob turning before the lock and the onsetof the knob locking. The subjective clarity scores in these cases

Figure 6. Subjective rating scores of clarity and event recognition rates.
(a) Slip down of manipulated object, (b) door knob turning.

218

were low as well. It is supposed that the increasing rate of thetangential force was insufficient because of the insufficient elastic
coefficient of VC. However, the parameters of VC are limited byseveral conditions, such as simulation stability, especially in thecase where VC is used with an ungrounded force display device
that allows finger penetration into the virtual object. Any method
to enhance the state change needs to be introduced for clarity to
improve.
The weakness of the tangential force sense in the knob turning
task was pointed out in the free interview with the subjects. The
supposed reason is the insufficiency of the VPS position
displacement. However, the distance between the pins was set tothe maximum distance that induces VPS. Therefore, more pins are
required to stimulate a wider area of the fingertip. A thinner VPS
display device is also to be developed.
The feasibility to display multi-degree-of-freedom force andslip and to assist the object manipulation tasks has been
demonstrated. More tasks of different types are to be examined
toward practical multi-fingered manipulation, such as a task that
requires the perception of the continuous change of tangential
force, one that needs to distinguish slip velocity, one that requires
the recognition of the direction of the reaction force vector, and soon.
6

CONCLUSIONSThis paper proposed a substitutive multi-degree-of-freedom force
display method that utilizes VPS. A multi-fingered virtual object
manipulating system based on the proposed method wasprototyped. The feasibility of the perceptional assistance ofcontact state change such as slip down of the manipulated object
and reaction torque induced by the rotation of the object-grasping
hand was verified through the virtual object manipulation tasks.
The future works are to improve the clarity of the state change andevaluate the task assistance effect in other manipulation tasks.
REFERENCES[1]

Y. Kohno, S. Walairacht, S. Hasegawa, Y. Koike, M. Sato:
Evaluation of Two-Handed Multi-Finger Haptic Device SPIDAR-8;
ICAT2001, pp.135-140, 2001
[2] G. Inaba, K. Fujita: A Pseudo-Force-Feedback Device by Fingertip
Tightening for Multi-Finger Object Manipulation; Proc. Eurohaptics
2006, Paris, pp.275-278, 2006
[3] R. Scheibe, M. Moehring, B. Froehlich: Tactile feedback at thefinger tips for improved direct interaction in immersive
environments; proc. IEEE Symp. on 3D User Interfaces, Charlotte,
pp.123–130, 2007
[4] E. Piateski, L. Jones: Vibrotactile pattern recognition on the arm andtorso; Proc. WorldHaptics, Pisa, pp.90-95, 2005.
[5] http://www.immersion.com/3d/products/cyber_touch.php[6] C. Kim, C. Kim, G. Yang, T. Yang, B. Han, S. Kang, D. Kwon:
Small and Lightweight Tactile Display (SaLT) and Its Application;
Proc. Eurohaptics 2009, Salt Lake City, pp.69-74, 2009[7] A. Yamamoto, S. Nagasawa, H. Yamamoto, T. Higuchi:Electrostatic Tactile Display with Thin Film Slider and Its
Application to Tactile Tele-Presentation Systems, IEEE Trans. Vis.
and Comp. Graph., vol. 12, no. 2, pp.168-177, 2006
[8] H. Regenbrecht, J. Hauber, R. Schoenfelder., A. Maegerlein: Virtual
Reality Aided Assembly with Directional Vibro-Tactile Feedback;Proc. Graphite 2005, Dunedin, pp.381-387 , 2005
[9] Y. Mizukami. H. Sawada; Tactile information transmission byapparent movement phenomenon using shape-memory alloy device:
Int. J. on Disability and Human Development, vol.5, No.3, pp.277284, 2006[10] R. S. Johansson , J. R. Flanagan: Coding and use of tactile signals
from the fingertips in object manipulation tasks; Nature Reviews
Neuroscience 10, pp.345-359, 2009