Interactive representation of virtual object in hand-held box
by finger-worn haptic display
Kouta Minamizawa 1

Souichiro Fukamachi2

Naoki Kawakami3

Susumu Tachi4

School of Information Science and Technology
The University of Tokyo
ABSTRACTTo deliver a realistic presence of virtual objects with a simple
haptic display in an augmented reality system, we have developed
a wearable haptic display to present the sensation of weight andinertial force of the virtual objects. In this study, we developed anaugmented reality application for our haptic device that represents
the dynamics of the virtual objects inside a real box based on aphysical simulation. We implemented the prototype system torepresent a virtual ball in a real box, and evaluated the capability
Of our proposed method. of our proposed method.

Figure 1. Conceptual picture of our proposed system.
By wearing our proposed devices, the user can feel theforce feedback of the virtual objects in a real box.

Keywords: Haptic I/O, interaction techniques, augmented reality.INTRODUCTION
In order to identify the contents of a box or a bottle without
opening it, one might hold and shake the box or the bottle andrecognize the contents by sensing their weight and inertial force.
In a virtual reality or augmented reality system, this type of
behaviour is required in order to improve the recognition of thevirtual objects. Recently, some researchers aimed to deliver thesensation of weight or the internal dynamics of a virtual object bymeans of wearable haptic devices. A portable haptic device
producing a force and a torque [1] illustrated that an ungroundeddevice was able to represent the internal force. A few devices
could in fact shift their spindle in order to deliver the movementof a barycentre [2]. Impact forces and the resulting vibrations
among the virtual objects are also important in order to be able torecognize these objects. Yao [3] demonstrated that the rotationand friction generated when moving an object can be reproduced
by recording and playing the impact forces and vibrations. In thisstudy, we tried to represent the virtual objects inside a real object
with the help of a wearable haptic display developed by theauthors [4]. Interactivity among operators and a real-time
performance are important in order for the operators perceive a
comfortable force feedback according to their movements.
Moreover, the shape, mass, volume, hardness and otherparameters of the internal objects are required to be easily/adjustable. Therefore, we employed a simple physical simulationthat calculates the vertical and shearing forces generated on thefingerpads, these forces are then reproduced by our proposed
haptic devices and relayed to the operator.

1e-mai: kout_minaizawa@pc.i.utokyoac.jp

le-mail: kouta minamizawa(0ipc.i.u-tokyo.acjp*2e-mail: souichiro_fukamachi@ipc.i.u-tokyo.acjp*3e-mail: kawakami@star.t.u-tokyo.acjp~4e-mail: tachi@star.t.u-tokyo.ac.jp

2METHOD

Finger-worn haptic display
In our previous research [41, we proposed a simple wearable
haptic display that represents the mass of a virtual object.
Conventionally, it is believed that it is necessary to reproduce theproprioceptive sensation in order to generate a weight sensation,and thus, a large grounded device is required. However, it was
found that the vertical and shearing forces generated due to thedeformation of the fingerpads can reproduce reliable weightsensations even when the proprioceptive sensations on the wrist
and the arm are absent. Based on this observation, we designed
the mechanism of the haptic display, as shown in Figure 2, which
has a simple structure comprising dual motors. Based on thismechanism, we implemented the prototype device shown inFigure 1. By wearing our proposed devices on the index finger
and the thumb, the operator could perceive the weight and theinertial mass of the virtual object in his/her hand during variedbehaviors.
2.1

rotate in opposite direction

rotate in same direction-operators

erUca
forer
generatedfingervertical force

operators

fingergenerated

shearing force

Figure 2. Our proposed method for generating vertical stress(left) and shearing stress (right) on fingerpad.[4]Symposium on Haptic Interfaces for Virtual
Environments and Teleoperator Systems 2008
13-14 March, Reno, Nevada, USA
978-1 -4244-2005-6iO8/$25.00 ©C2008 IEEE367

2.2
Representation of internal virtual object
To reproduce the deformation of the fingerpad when grasping abox with virtual objects inside it, we developed a simulationbased force feedback system using the following method. We
used the open dynamics engine (ODE) [5] as a physical simulator.
1) Set the model of the box, the virtual fingers, and the virtual
objects in the physical simulator.
2) Fix a three-dimensional accelerometer on the real box.
3) Assign a sign-reversal acceleration to the virtual objects. The
virtual objects start moving with this acceleration.
4) Calculate the force vector on the virtual fingers generated dueto the inertial mass of the virtual objects.
5) Generate the haptic devices to reproduce the calculated
vertical and shearing forces.

75% DLCDCD< 0.5

n 0.250'/--lStandard stimulus = 50 g
°40
-20
0+20+40
Relative weight of the comparison stimulus
to the standard stimulus [g]Figure 4. Average rate of "heavier" responses for a 50-g ball as

the standard stimulus. The solid line indicates the PSE and3EXPERIMENT
The method of constant stimuli was employed for thisexperiment wherein three subjects (aged 23-25) were asked towear the haptic displays on their index finger and thumb and holda light-weight box. As a standard stimulus, the weight of a
specific virtual ball (as shown in Fig. 3-a) was represented by theproposed method for a few seconds. Subsequently, another virtualball with a random weight (ranging from 1 g to 300 g) wasrepresented as a test stimulus. The subjects could move their arms
freely and the inertial mass of the virtual objects were represented
in these trials. The subjects then stated whether the weight of thevirtual ball in the test stimulus was "heavier" or "not heavier" incomparison to that in the standard stimulus according to a twoalternative forced-choice procedure. The cycle length of thephysical simulation was 5 ms (200 Hz), and the cycle length of themotor control was 500 pts (2 KHz).
Figure 4 shows the average rate of "heavier" responses obtained
during the trials with a standard stimulus of 50 g. The circles
represent the average of all the 16 trials conducted with eachsubject. The curve indicates the fitted line with a cumulative
normal distribution. The solid line indicates the point of subjective
equality (PSE), and the dashed lines indicate the 75% difference
limen (DL) that was derived from the difference between the PSE
and the 75% discrimination threshold. Table 1 presents the 75%
DL for each standard stimulus. This result confirmed that our
proposed method for reproducing the stresses on the fingerpads
provides a clear perception for recognizing the weight and inertial
force of the virtual object. The recognition resolution in the 10-gstandard stimulus trials was poorer than that in the other trials
because the device (45 g on each finger) and the real box (30 g)
were constantly placed on the subjects' fingers, and the 10-gstandard stimulus on the fingers was insignificant in comparison
with these constant weights. It is believed that decreasing thedevice weight will result in a further dexterous representation of
the virtual mass.

0 75

the dashed lines indicate the 75% DL
Table 1. 75% DL for four types of standard stimuli in representing
a virtual ball in a hand-held real box

4

standard stimulus!750DL
10 gf
4.3 gf(43.0%)
50 gf
10.7 gf(21.4%)
100 gf21.0 gf(21.0%)
200 gf58.2 gf(29.1%)0)
CONCLUSION & FUTURE WORKIn this paper, we have proposed a simple simulation-based
method to represent the mass of virtual objects inside a real object.
Using our proposed finger-worn haptic display, we haveinteractively developed a haptic feedback system that represents
the mass of a virtual ball in a real box. The experiment results
indicate that the subjects could recognize the weight of theinternal virtual object using this method.
We have also confirmed that more than one internal virtualobject, as shown in Fig. 3-bc, can be represented using ourproposed method in a technical demonstration [6]. The haptic
feedback in this system is so sensitive that the small forces ofcollisions among the virtual balls can also be recognized. Whileconventional haptic displays can barely represent such a sensitive
sensation, our proposed method can represent more realistic
sensations of various internal objects. In future works, we will
represent other types of virtual matters such as liquids andanimate beings.
REFERENCES[1] N. Nakamura and Y. Fukui, "Development of a Force and TorqueHybrid Display," in Proc. WorldHaptics 2005, pp. 633-634, 2005.
[2] H. Kuribayashi, S. Nakamura, Y. Kume, "A Hand-held Display
Presenting Visual and Force Information," in Proc. WorldHaptics
2007, pp. 576-577, 2007.
[3] H. Y.' Yao and V. Hayward, "An Experiment on Length Perception

with a Virtual Rolling Stone," in Proc. EuroHaptics Int. Conf. 2006,
pp. 275-278, 2006.
[4] K. Minamizawa, H. Kajimoto, N. Kawakami, S. Tachi, "WearableHaptic Display to Present Gravity Sensation," in Proc. WorldHaptics__2007, pp. 133-138, 2007.
http://www.ode.org/~~~~~~~~~~~~~~[5]_abFigure 3. Physical simulation modes: virtual balls in a box

368

c[6] K. Minamizawa, S. Fukamachi, H. Kajimoto, N. Kawakami, S. Tachi,"Gravity Grabber: Wearable Haptic Display to present Virtual Mass
Sensation," ACM SIGGRAPH 2007, Emerging Technologies, 2007.