Effect of haptic feedback on pseudo-haptic feedback for arm display
Mari Tatezono1, Katsunari Sato1, Kouta Minamizawa1, Hideaki Nii2, Naoki Kawakami1 and Susumu Tachi2
1

Graduate school of Information Science and Technology, The University of Tokyo, Tokyo, Japan(Tel : +81-3-5841-6917;
E-mail: {Mari_Tatezono, Katsunari_Sato, Kouta_Minamizawa, Naoki_Kawakami}@ipc.i.u-tokyo.ac.jp)
2
Graduate school of Media Design, Keio University, Yokohama, Japan(E-mail: {nii, tachi}@tachilab.org)
Abstract: We propose a haptic feedback system for an arm in order to obtain the sensation of stiffness and the boundaryof virtual objects. Powerful haptic displays are large and heavy, interfering with movement in the virtual environment.
Here, we focus on an illusion called pseudo-haptic feedback, which provides the results of haptic feedback using only
visual impressions. Since pseudo-haptic feedback is known to be inappropriate for applications requiring strong force, acombination of pseudo-haptic feedback and real haptic feedback is proposed for a compact haptic display. This study
examines the effectiveness of this combination by comparing the stiffness of two walls. Pseudo-haptic and real haptic
feedbacks are applied to one wall, and only real haptic feedback is applied to the other wall. The results verify that themagnitude of the force perceived with the application of a combination of pseudo-haptic and real haptic feedbacks isgreater than that of the force perceived by the application of only real haptic feedback.
Keywords: pseudo-haptic, haptic feedback, virtual reality.

1. INTRODUCTION
Virtual reality technology provides us with thesensation of walking into a computer graphic
environment and actually operating an object in it.
Haptic feedback assists us in manipulating a virtual
object. If there is no haptic feedback, we have todetermine whether we have touched an object or not bysight alone. If our hand is visualized along with theobject, the reality is spoiled. Therefore, haptic feedback
is important for virtual reality technology.
Various haptic displays have been previously
proposed. In this study, a haptic display for an arm withthe sensation of softness and boundary, which is thought
to be essential for interaction with virtual objects, isconsidered. Since the fingers and palms of a hand aremost often used when an object is touched, most of theexisting haptic displays focus on them. However, ineveryday life, other parts of the body are also used totouch objects. For example, the hand and arm are used
to touch a large object such as a soft toy. Therefore, inthis case, a haptic display using the arm would befunctional (as in Fig. 1). It is desirable to move freely inthe virtual environment; therefore, a haptic display
should not restrict the workspace or interfere withbodily movements.
There are many methods for providing haptic
feedback to arms. For example, methods that involvethe direct application of force to an arm, production ofsymbolic tactile feedback, direct stimulation of a muscle
or a nerve, and use of visual feedback have been
proposed. Litier et al. [1] presented a 7-DOF (Degree ofFreedom) portable exoskeleton with DC motors andsensors, and Furisoli et al. [2] proposed a 5-DOF groundexoskeleton with DC torque motors and carbon fiber
structural parts. These displays are placed on the arm,and force is directly applied when a virtual object istouched. These displays can produce a large force.
However, the larger the force these displays apply, thelarger and heavier the devices become. A large andheavy device restricts workspace and interferes withbodily movements. Bloomfield et al. [3] proposed a
display with vibratory tactors. This display provides
only a symbolic tactile feedback, not a force feedback.
An example of a method that directly stimulates amuscle or a nerve is FES (functional electricalstimulation). Fujita et al. [4] proposed a method forcontrolling the motion of a paralyzed upper limb using
FES and a neural network. However, FES does not seemto be good for haptic feedback, because an invasive
display is difficult to use casually, and stick-on
electrodes cannot produce a constant sensation. An
example of a method that involves the use of visualfeedback is a pseudo-haptic approach. Pseudo-haptic
feedback does not need a force display. However, it is
known that pseudo-haptic feedback varies with eachindividual, and it may be difficult to produce a large
force using it. The abovementioned haptic methods
seem ineffective for obtaining desired results.

Fig. 1 Illustration of an interaction with a virtual object.

A compact haptic display using both pseudo-haptic
and real haptic feedback is proposed in this study. (Aforce that is directly applied to the arms is referred to as“real haptic” in this paper). It is expected that the
proposed haptic display will produce a larger force thanthe display using only real haptic feedback, and thetarget value of the real haptic feedback will become
small along with the display. The effectiveness of a real
haptic and pseudo-haptic combination is examined toshow its benefits.

2. METHOD OF COMBINATION
The proposed combination of pseudo-haptic andhaptic feedback for a compact haptic display is shownin Fig. 2.

Fig. 2 Image of the proposed method.
2.1 Real haptic feedback
A display using real haptic feedback provides
stiffness based on the positions of the user’s hand andthe object. In accordance with Hooke’s law (Eq. (1)),
the force (Fd) that the device applies to a user is related
to the stiffness (kd) of the device and the displacementof the user’s arm from the boundary of the object (xd).

Fd = k d x d(1)

Displays using real haptic feedback are mainly
categorized into the following two types: portable andground-based displays. These displays can produce
large forces. However, motors that are needed to
produce these large forces tend to be cumbersome.
Therefore, displays that are capable of producing a large
force become heavy and awkward, interfering with the
movement of the body. It becomes difficult to move thearm upward or rapidly and to walk around and move thearm carrying the weight of the device. On the other
hand, the ground-based display is lighter than the
portable display, because the ground-based display ishung from a roof, wall, or supporting post. However, theground-based display still restricts workspace andinterferes with movement.

2.2 Pseudo-haptic feedback
Pseudo-haptic feedback is a well-known illusion,
where haptic feedback is obtained by using only visualfeedback. For example, when we constantly manipulate
a mouse but the cursor moves slowly, we feel a bumpy
surface [5]. There have been studies on the perceptionof stiffness and friction using this phenomenon.
The force (Fp) that the pseudo-haptic feedback
applies to a user can be assumed to be related to thestiffness (kp) of a virtual object and the displacement(xp) of the virtual arm from the boundary of a virtual
object (Eq. (2)) [4]. xp is related to the proportionality
coefficient (α) and xd (Eq. (3)). The value of kp seems todepend on the user, in other words, in pseudo-haptics, kp
is a value of individual difference. It appears that themagnitude of Fp can be changed by changing the valueof α.

Fp = k p x p(2)x p = α xd

(3)

In contrast, when a virtual object is not touched, thedisplacement (x’p) of the virtual arm is equal to thedisplacement (x’d) of the user’s arm.
Pseudo-haptic feedback does not need a mechanicalhaptic display. Therefore, the body can be moved freely.
However, considering the force the pseudo-haptic
feedback can apply, there are questions whether
pseudo-haptics can produce an adequate force. Lecuyer
et al. [6] showed that we feel something is wrong
according to the ratio between the user’s displacementof the input interface and the visual displacement of theobject in the virtual environment. When the difference
between the visual information and the sensation of
proprioception becomes large, pseudo-haptics will notwork normally. However, this ratio becomes large when
we attempt to apply a large force using pseudo-haptic
feedback. Therefore, it appears to be difficult to apply alarge force using pseudo-haptic feedback. A small force,
which can be applied using pseudo-haptic feedback, isinadequate for our purpose. Another known defect of
pseudo-haptic feedback is that some people only
slightly perceive the feedback.
2.3 Combination of real haptic and pseudo-haptic
feedback
As previously mentioned, both real haptic feedback
and pseudo-haptic feedback have advantages anddisadvantages. To compensate for the disadvantages ofeach type of feedback, a combination of pseudo-haptic
and real haptic feedback is proposed for a compact
haptic display.
It appears to be difficult to apply a large force using
pseudo-haptic feedback. However, helpful experimental
facts can be extracted from previous work. Paljic et al.
[7] observed that pseudo-haptic feedback is enhanced
by using an elastic interface as opposed to using
isotonic interfaces, such as a mouse. It is considered thatthe resistance of the device produces real force feedback
and enhances the pseudo-haptic sensation. Therefore, it
is considered that pseudo-haptic feedback appears to beenhanced when used in combination with real haptic
feedback. If pseudo-haptic feedback can be enhanced, it
may apply a larger force so that our sensation of touchallows us to more strongly perceive stiffer objects aswell as the boundaries of objects.
In the combination of real haptic and pseudo-haptic
feedbacks, the force (Fu) that the user perceives appears
to be expressed as a function of Fd and Fp (Eq. (4)).
Large Fu can be achieved by increasing Fd. However,since our objective to develop is a compact haptic
display, a small Fd and a large Fp are desired.

Fu = f (Fd , F p )makes when the subjects manipulate it, the subjects
listened to white noise with headphones during theexperiment. The subjects moved a red ball (pointer)right and left on the screen by manipulating PHANToM
and touching the virtual wall (the white area on thecomputer screen).

(4)

When both real haptic and pseudo-haptic feedbacks areused, the target value of Fd is probably smaller than thatwhen only real haptic feedback is used. Therefore, itwould be possible to develop a compact haptic display
using both pseudo-haptic and real haptic feedbacks.
There are some questions about the function of Fu.
Firstly, is pseudo-haptic feedback produced when
pseudo-haptic feedback is combined with real haptic
feedback? Pseudo-haptic feedback is an illusion of
haptics caused by visual information. It is thought that
we cannot perceive pseudo-haptic feedback when weapply pseudo-haptic feedback with real haptic feedback.
In the worst case, it is considered that the pseudo-haptic
feedback decreases the magnitude of the perceived force
applied using the real haptic feedback. Secondly, howlarge is the pseudo-haptic feedback? If the magnitude of
the pseudo-haptic feedback is very small, it cannot be
expected that the magnitude of the perceived force will
be enhanced by the combination of pseudo-haptic andreal haptic feedbacks. Thirdly, does the magnitude of
the pseudo-haptic feedback change depending on themagnitude of the real haptic feedback? If the value ofthe real haptic feedback that efficiently enhances themagnitude of the pseudo-haptic feedback is known, it
will be valuable in deciding the target value of the real
haptic feedback that should be applied to the haptic
display. Therefore, an experiment was performed toprovide the answers to these questions.

3. EXPERIMENTS
In this experiment, the effect of the perception offorce was examined when pseudo-haptic feedback wasapplied with real haptic feedback. The Fp and Fd
conditions were compared with the Fd only condition.
3.1 Experimental procedure
Fig. 3 and Fig. 4 show the experimental setup andvisual information that was applied to the experimental
subjects. PHANToM (SensAble Technologies Inc.) wasused as a haptic display for applying real haptic
feedback and determining the user’s hand movements.
To eliminate the effect of the sound that PHANToM

Fig. 3 Experimental set-up.

Fig. 4 Visual information in the experiment.
The magnitudes of Fu and Fp were quantified bycomparing them to the magnitude of the force that
PHANToM applied to the subjects. The psychophysicalexperiment performed was an up-and-down method (itis sometimes called a stair-case method). In this method,
there are two series—an ascending series and adescending series. The subjects were instructed to move
PHANToM to the left and touch the virtual wall A while
pseudo-haptic feedback with real haptic feedback or
only real haptic feedback was applied. In thisexperiment, the magnitude of real haptic feedback was
not affected by the distance that a pointer presses into
the wall. The magnitude of the pseudo-haptic feedback
was constant in order to determine the magnitude of thepseudo-haptic feedback than the stiffness. First, thevalue of the real haptic feedback was 0. Then, thesubjects were instructed to touch the virtual wall B
while only real haptic feedback was applied forreference. First, the value of the real haptic feedback
was 0 for reference. The subjects were supplied withvisual information with no pseudo-haptic feedback
when they touched wall B.
The subjects were instructed to choose the stiffer wall
between the two virtual walls displayed on the screen
and indicate their answer by typing “A” or “B” on the
keyboard. If the subject chose “A,” the magnitude of thereal haptic feedback was increased by 0.1 [N] forreference. The comparing and increasing was repeateduntil the subject chose “B.” This process was an
ascending series. If the subject chose “B,” the value ofthe real haptic feedback was changed to 3.3 [N] forreference, and the stiffness of the two walls wascompared. If the subject chose “B,” the magnitude of
the real haptic feedback was decreased by 0.1 [N] forreference. The comparing and decreasing was repeateduntil the subject chose “A.” This process was adescending series. The value of real haptic feedback wasincreased by 0.5 [N] when the descending series ended,and the subject started an ascending series with a newreal haptic feedback value.
In this experiment, the magnitude of thepseudo-haptic feedback was constant, and the value of α
was 0.2. Each subject performed these two series for thefollowing five values of real haptic feedback for wall A:
0, 0.5, 1.0, 1.5, and 2.0 [N]. The subjects repeated thistrial two more times. The subjects who took part in thisexperiment included 5 men, aged 22–28. All thesubjects were right handed, and they used their
dominant hand to determine the stiffer wall. During
each trial, the subjects were allowed to test each wall as
many times as they wanted.
To eliminate the effect of habituation, the subjects
conducted training trials, which included an ascending
series and a descending series, for two values of real
haptic feedback.
3.2 Result and discussion
Fig. 5 shows the effect of combining real haptic
feedback with pseudo-haptic feedback. The graph in thisfigure shows the values for each subject (a–e) as well as
an average of the 5 subjects (f). The horizontal axis
denotes the magnitude of the real haptic feedback forwall A, and the vertical axis denotes the magnitude of
the real haptic feedback for wall B. Therefore, thevertical axis shows the magnitude of the force thesubjects perceived (Fu). The vertical bar indicates thestandard deviation in graph (f). The green line shows thesimulation magnitude of Fu when the subjects were
provided with real haptic feedback. The blue line showsthe experimental results of the magnitude of Fu when
the subjects were provided with only real haptic
feedback. The red line shows the experimental results ofthe magnitude of Fu when the subjects were provided
with a combination of real haptic feedback andpseudo-haptic feedback.
The red line is above the blue line for each value ofreal haptic feedback for wall A for subjects A, B, and C.
This shows that pseudo-haptic feedback is certainly

perceived even under the condition of a combination ofpseudo-haptic feedback and real haptic feedback. Thisimplies that the combination of pseudo-haptic feedback
and real haptic feedback extends the maximum output
value of the haptic display. It may be possible todevelop a haptic display that is small and light by
combining pseudo-haptic feedback and real haptic
feedback. It is interesting to note that pseudo-haptic
feedback can be perceived without being reversed withreal haptic feedback, even when a large real haptic
feedback is applied.
There is a possibility that the magnitude of thepseudo-haptic feedback changes depending on themagnitude of the real haptic feedback. As shown in Fig.
5 (f), the magnitude of the pseudo-haptic feedback isapproximately 0.15 [N] under the experimental
condition. Furthermore, the magnitude of thepseudo-haptic feedback is approximately 0.25 [N] whenthe magnitude of the real haptic feedback is 1.5 [N], and
0.4 [N] when the magnitude of the real haptic feedback
is 2.0 [N]. This result indicates a possible connection
between the magnitude of the real haptic feedback andthat of the pseudo-haptic feedback. This trend is clearlyshown in Fig. 5 (b).
The magnitude of the pseudo-haptic feedback without
the real haptic feedback is small. This result confirms
the findings of Lecuyer et al., because the ratio between
the user’s displacement of the input interface(PHANToM’s
displacement)
andthevisualdisplacement of the object in the virtual environment(the red ball’s displacement) become large when only
pseudo-haptic feedback is applied. However, thefindings are not confirmed when the magnitude of real
haptic feedback becomes large. This is because only
five values of real haptic feedback were used for wall A
up to 2.0 [N] in this experiment, and the ratio of the two
displacements may not become so large. We areinterested in examining how the effect of the value ofreal haptic feedback on the value of pseudo-haptic
feedback changes with larger values of real haptic
feedback.
Pseudo-haptic perceptions differ from person toperson. In this experiment, subject B perceived
pseudo-haptic feedback only slightly without real haptic
feedback. However, he perceived pseudo-haptic
feedback more strongly along with real haptic feedback.
If the ratio of people who can perceive pseudo-haptic
feedback increases with the use of real haptic feedback,it implies that a display with a combination ofpseudo-haptic feedback and real haptic feedback may bebeneficial for more people than those with only
pseudo-haptic feedback. Therefore, the occurrence
probability of pseudo-haptic feedback depending on the
addition of real haptic feedback is significant.
In this experiment, the parameter of pseudo-haptic
feedback was constant (α = 0.2). Therefore, examinationof the variance of pseudo-haptic feedback is also needed.
If we can adjust magnitude of pseudo-haptic feedback inhigh resolution, a haptic display with a large range of
output force and high resolution may be realized, by

(a) subject A(d) subject D(b) subject B(e) subject E(c) subject C(f) average of all subjects

Fig. 5 Effects of the real haptic feedback on pseudo-haptic feedback.
combining a haptic feedback that has a large range of
output force and low resolution and a pseudo-haptic
feedback that has a narrow range of output force andhigh resolution. Furthermore, it is interesting todetermine whether pseudo-haptic feedback applies toforces in a negative direction. By changing theparameters of the pseudo-haptic feedback, we may
perceive a smaller force than that perceived with real
haptic.
This experiment shows that pseudo-haptic feedback isperceived even when pseudo-haptic feedback and real
haptic feedback are applied. When α = 0.2, themagnitude of pseudo-haptic feedback is approximately
0.15 [N] when only pseudo-haptic feedback is applied,
and it is approximately in the range of 0.2–0.4 [N] when
pseudo-haptic and real haptic feedbacks are applied.

The magnitude of pseudo-haptic feedback is nearly 10
percent of the magnitude of real haptic feedback, whichis smaller than we expected. This method appears to fallshort of our needs for realizing a compact haptic display,although it may be modestly beneficial. However, themagnitude of the force we perceive certainly becomeslarger by combining pseudo-haptic feedback and real
haptic feedback, and it would appear that this method
can be used as an enhancement to real haptic feedback.
When we applied a very large real haptic feedback, it
was considered that the proportion of the pseudo-haptic
feedback to the real haptic feedback would become
small. However, this method may be effective in cases
where the touched object has a particular stiffness. Thatcase would occur when the magnitude is such that the
proportion of the pseudo-haptic feedback to the real

haptic feedback is very large. The experiments in thisstudy were performed in a determinate condition inwhich the parameter of the pseudo-haptic feedback was
constant (α = 0.2), and the values of the real haptic
feedback were 0–2.0 [N]. Future studies will include
experiments in which the value of α changes and thevalue of the real haptic feedback is greater than 2.0 [N].

4. CONCLUSIONSThis study focused on a haptic display for arms. A
strong force is required to perceive the sensation of
stiffness and the boundary of virtual objects. Therefore,
the haptic display becomes large and heavy. On theother hand, the application of a strong force using only
pseudo-haptic feedback appears to be difficult.
Therefore, we examined the effectiveness of a compact
haptic display using a combination of pseudo-haptic
feedback and real haptic feedback. In the results,
subjects perceived pseudo-haptic feedback when it wasapplied along with real haptic feedback. Therefore, aforce of approximately 0.2–0.4 [N] more was perceived
by combining pseudo-haptic feedback and real haptic
feedback than when only real haptic feedback wasapplied. The magnitude of the difference is nearly 10
percent of the magnitude of real haptic feedback.
Similar experiments will be performed under thecondition where the value of α changes and the value ofreal haptic feedback is greater than 2.0 [N].

ACKNOWLEDGMENT
This work was partly supported by SCOPE: Strategic
Information and Communications R&D Promotion
Programme.

REFERENCES[1] P. Letier, M. Avraam, S. Veillerette, M.Horodinca, M.
De Bartolomei, A. Schiele and A. Preumont, “SAM : A
7-DOF portable arm exoskeleton with local joint
control.” Intelligent Robots and Systems, pp. 3501-3506,
2008.
[2] A. Frisoli, F. Rocchi, S. Marcheschi, A. Dettori, F.
Salsedo and M. Bergamasco, “A new force-feedback
arm exoskeleton for haptic interaction in virtualenvironments.” World haptics 2005, pp. 195-201,
2005.
[3] A. Bloomfield and N. Badler, “Collision Awarenessusing Vibrotactile Arrays.” IEEE Virtual Reality
Conference, 163-170, 2007.
[4] K. Fujita, H. Takahashi, “Attempt for Learning
Control of Paralyzed Upper Limb Motion by FES using
Neural Network.” JSME annual meeting 2000,
pp.365-366, 2000.
[5] A.Lecuyer, J. Murkhardt and L. Etienne. “Feeling
Bumps and Holes without a Haptic Interface : thePerception of Pseudo-Haptic Textures.” CHI 2004, pp.
239-246, 2004.
[6] A. Lecuyer. “Simulating Haptic Feedback Using
Vision: A Survey of Research and Applications ofPseudo-Haptic Feedback.” Presence, Vol.18, No.1, pp.
39-53, 2009.
[7] A.Paljic, J.-M. Burkhardt and S. Coquillart.
“Evaluation of pseudo-haptic feedback for simulating
torque: a comparison between isometric and elastic
input devices.” Haptic Symposium 2004, 2004.