A Device and Method for Multimodal Haptic
Rendering of Volumetric Stiffness
Yon Visell1? , Keerthi Adithya Duraikkannan1 , and Vincent Hayward2
1

Drexel University, Dept. of Electrical and Computer Eng., Philadelphia, USA
2 Sorbonne Universités, UPMC Univ. Paris 06, ISIR, Paris, France

Abstract. Vibrotactile signals are produced during haptic exploration of compressible objects through a variety of contact and bulk mechanical processes.
Prior studies have found that vibrotactile feedback can influence stiffness perception, but the reason for this is unknown. Here, we investigated the role ofvibration in stiffness perception during object squeezing. We propose a physically motivated explanatory model and rendering algorithm relating vibrotactile
and force-displacement cues, then present a novel haptic interface that was designed to accurately reproduce these cues. Finally, we present the results of anexperiment on the perceptual integration of vibrotactile and force-displacement
cues during one- and two-finger stiffness perception. The results indicate that vibration feedback can increase perceived object softness during interaction with
one finger, but preclude a similar conclusion for two-finger grasping. We argue
that the results support the proposed model once innate differences in one- andtwo-finger exploration are accounted for.

1

Introduction

Mechanical vibrations are produced through a diverse array of contact and bulk mechanical interactions, including sliding friction, transient contact between objects, orthe inelastic deformation of inhomogeneous media (e.g., a bag of coffee beans). The
vibrotactile sense is capable of assisting the brain in discriminating touched surfaces
or felt objects, in extracting properties such as roughness or surface regularity, and in
identifying events, such as changes of contact conditions [1–3]. However, it has rarely
been considered as a potential cue to softness, since soft or deformable objects or structures are rarely thought of as sources of such vibrations. Nonetheless, in many cases,
such vibrations are often associated with macroscopic displacements of surface or bulk
elements, leading to correlations between vibromechanical energy production in a bulk
medium and macroscopic constitutive (force-displacement) behavior. Indeed, several
prior studies have suggested that action-synchronized vibrotactile feedback can strongly
influence haptic softness perception during object palpation [4–6]. However, the mechanism by which this occurs is uncertain.
The goal of our investigation was to measure empirically the role of vibromechanical sensory information in the perception of haptic object stiffness during grasping.
We first propose an explanatory model and rendering algorithm relating vibrotactile
and force-displacement cues. We then present a novel haptic interface capable of reproducing these cues during multi-finger squeezing, over a wider dynamic and frequency? email: yon.visell@drexel.edu

2

Yon Visell, Keerthi Adithya Duraikkannan, Vincent Haywardrange than has been possible with previous devices. Finally, we present an experimentevaluating this effect, the results of which lend some support to the proposed model.
1.1

Vibrotactile Rendering of Volumetric Stiffness: A Mechanical Model

The compression of an object or mechanism can result in both macroscopic stress andstrain signals and higher frequency vibrations generated through inelastic contact orbulk mechanical processes. We can examine a prototypical physical model encompassing such effects in order to derive a relation between vibrotactile and force-displacement
stimuli. Here, we consider an elementary model for inelastic processes in a compressedbulk medium (Fig. 1).
Action phase dependence The linear viscoelastic part of the strain response of a compressed medium x(t) due to an applied force F(t) can be modeled as a mechanical
impedance Z with F(s) = Zc (s)v(s), where s is the Laplace transform variable and v(t)is the compression rate. If we also assume that there is a linear dynamical dependence
between an increment dx(t) = v(t)dt of compression and an increment r(t)dt of inelastically produced vibration noise, then we obtain an action-phase dependence of theform r(s) = Zn (s)v(s), where r(t) is the amplitude of vibration noise, and the vibration
production impedance Zn (s) is introduced to encode stress in the sliding medium andand other properties of the compressed material that affect noise generation.
Rendering the spectrum of mechanical vibrations To model vibration noise delivered to the finger during shear inelastic or frictional deformation, we refer to priorstudies of force fluctuations in the frictional sliding of a finger on a surface [7], thecompression of disordered media [8], and many other physical systems, all of which
are observed to elicit random forces δ F(t) with fluctuation power spectra having characteristic 1/ f dependences on frequency f . Such fluctuations can be modeled as pinknoise p(t), i.e., a random process with power spectrum density S( f ) ∼ 1/ f .
Mechanically, the finger acts as a damper above 100 Hz, obeying a constitutive law
Fskin (t) = −Bẋskin (t), where B is a damping coefficient, implying p(t) = −Bẋskin (t).
Now, most haptic devices can be approximated by an inertia m, which is much greaterthan that of the finger pad. Thus, if we command a current I(t) yielding a force F(t) =γI(t), where γ is a motor gain factor, we haveF(t) = γI(t) = mẍdevice (t) = mẍskin (t)AFB

Dynamic
envelope

C

F

Force

VibromechanicalVibration
acceleration,Stimulusa (t )

x
R1

R2

Force
Force
Displacement,x (t )

Dynamic

Dynamic
envelopeenvelope

Dynamic
envelope

Vibration
Vibration
Stimulus

Force
Force, F (tStimulus)
Vibration
Stimulus

Fig. 1. Compressing an object can involve both viscoelastic deformation (A), and the release ofvibromechanical energy (B, C) through a variety of processes, including shear sliding.

Vibrotactile Rendering of Volumetric Stiffness

3

In the case of interest, we aim to impose a displacement on the finger pad equal to that
excited by force fluctuations modeled as pink noise. From the foregoing, we can writeẋskin (t) = −p(t)/B, which yields
F(t) = mẍskin (t) =−m−mṗ(t) ⇒ δ Fskin (t) =
ṗ(t)
B
B

(1)

That is, the commanded force fluctuation should be proportional to the derivative of thepink noise process. We use this result to render vibromechanical stimuli for the skin thatpossess the vibration spectrum characteristic of the inelastic process of interest (e.g.,frictional compression). In our implementation, we synthesize pink noise in discrete
time by applying a 3 dB/octave lowpass filter H(s) to gaussian white noise n(s), sothat p(s) = H(s)n(s), then apply a differentiator filter D(s) ≈ s to the result. D(s) isband limited to limit gain and prevent aliasing, and is implemented in discrete time
as a FIR filter of order 128. The resulting commanded surface noise force signal isδ F(t) = D(s)H(s)n(s).
Multimodal rendering For the integrated rendering of haptic force-displacement stimuli and vibrotactile feedback, we model the former as a mechanical impedance Zc (s)describing the mean bulk constitutive (force-displacement) behavior F(s) = Zc (s)v(s),and the latter via an impedance model Zn (s) for the rate r(t) of vibromechanical energy production vs. compression rate, with r(s) = Zn (s)v(s). The fluctuating force is
non-zero only during loading, when it is given by
δ F(t) = A γ r(t) ṗ(t), r(t) = L −1 {Zn (s)v(s)}(2)

where γ is the actuator gain and L −1 denotes the inverse Laplace transform, see Fig. 2.
Due to the correlations between stiffness and vibration response embodied in thismodel, it is plausible that the latter provide a perceptual cue to stiffness. We conducted
stiffness experiments for one- and two-digit grasps, to assess this hypothesis.

2

The effect of vibration feedback on stiffness perception

We designed experiments to assess the relative contribution, if any, of the putative vibration and stiffness cues described above to the perceived softness of squeezed (virtual)objects. In them, participants were presented with stimuli that varied in stiffness and vibrotactile feedback amplitude. Participants performed perceptual judgements as to the+–
Fd=F+𝜹F
Amp,
Motor

𝜸

User

Fh

–
u = u + 𝜹u

v

Yp

F

End effector
–
u

Σ
𝜹u

Zc
.
p(t)

A

×

×

Zn

Fig. 2. A model for the haptic rendering of multimodal stiffness stimuli, incorporating both kinesthetic and vibrotactile cues.

4

Yon Visell, Keerthi Adithya Duraikkannan, Vincent Hayward

stiffness of the felt stimuli, from which we were able to infer the extent of contribution of vibration to stiffness perception. During stimulus exploration, participants usedeither two-digit precision grip or one finger pressing with the forearm supported.
2.1

Apparatus: The UGrip Haptic Display

We present a new haptic interface that we have designed for the present investigation.
We refer to it as the UGrip display. It is a force feedback device permitting its users tosqueeze virtual objects in a haptic simulation (Fig. 3). The device presents its user witha pair of manipulanda that can accurately display specified force-displacement relations
as well as high frequency vibromechanical feedback.
Mechanical structure and sensing The interface consists of a pair of anodized aluminum plates with rubberized surfaces that are squeezed in opposition by the index
finger and thumb. Each of the plates is able to displace up to 4 cm, so that up to 8
cm of compression of a virtual object can be simulated. The plates are coupled to low
noise linear guides, ensuring translation with low friction and, statically, high torsional
stiffness. Both actuators and sensors in the device are contactless. This results in good
control in the direction of motion and stiff coupling to actuators. By design, the device has trivial kinematics, simplifying control implementation and ensuring that thefrequency bandwidth extends to nearly 1 kHz. The displacement of each plate is sensed
with a non-contact magnetic encoder (LM13, R ENISHAW I NC ., I LLINOIS , USA) witha linear resolution of 1 µm.
Force display and calibration The apparatus is designed to reproduce forces that canstatically resist displacements of the fingers during pinch grasping, and concurrently topresent force- or displacement-dependent vibromechanical stimuli to one or two digits.
The linear voice coil actuators (M ODEL NCC15-24-050-1R, H2W T ECHNOLO GIES , S ANTA C LARITA , CA) are each able to reproduce high forces, i.e. more than 37
N of static or 100 N of peak force. They generate a force proportional to the driving current I, the number n of coil loops, the field strength constant B of the permanent magnet,
and a geometry factor Γ (x) equal to the proportion of permanent magnetic flux passing
through the coils for a given configuration (displacement) x, so that F(x) = nBIΓ (x). In

Fig. 3. The UGrip display device, presented here for the first time, is designed to enable its users tosqueeze virtual objects (A). The device (B, C) consists of a pair of electromechanically controlled
plates driven by high-power voice coil motors, yielding a display system capable of reflecting thefull range of forces and frequencies that the fingers are sensitive to.

5

40

3
0

30

Force (N)

-3
20
10

20

0
10

10

5

0
1
0
-1

0
1

0

Noise (N)

Position (mm)Force (N)Noise (N)Vibrotactile Rendering of Volumetric Stiffness

2.04 N/mm

0Time (s)

4

8

12

16

20

Displacement (mm)Fig. 4. Left: Time-domain stimulus components during a typical squeezing motion with the apparatus. Right: The rendered force-displacement profile is linear, and is accompanied by vibrationnoise. The amplitude of the latter is constant when the rate of loading is constant.

this device, actuators are driven by voltage-to-current mode amplifiers (H2W T ECH NOLOGIES , S ANTA C LARITA , CA) sufficient to produce the forces listed above. Forces
measurement and calibration was performed by instrumenting the end effectors with
precision miniature load cells (M ODEL LLB130, F UTEK , I RVINE , CA).
Force feedback control was effected using the controller shown in Fig. 3. It was implemented using a a two-axis motion controller (DMC-4123, G ALIL M OTION C ON TROL , ROCKLIN , CA). The programmable controller executes custom software implementing hard real time control loop with sampling frequency 8 kHz, ensuring artifactfree force reproduction. Program parameters are modified at runtime using ethernet
communication from a personal computer executing the experiment software.
Stimuli We rendered multimodal stiffness stimuli using the method presented above.
Partiipants explored the stimuli by squeezing the manipulanda with a two finger precision grip or by pressing it with one finger with the forearm supported (Fig. 6). Output
forces Fi , i = 1, 2 for each finger were computed as functions of measured positions xi .
The forces are given by an additive combination Fi = F i + δ Fi , where F i is the constitutive component, which varies slowly, and δ Fi is the fluctuating component. Due tothe inelastic nature of the latter, we presented vibrations only during loading. Vibration
stimuli were generated as shown in (Fig. 3). We model the noise generation impedanceZn (s) as equal, up to a constant factor, to the stiffness Zn (s) = Zc (s) = k/s so that thevibration stimulus is given by δ F(t) = Ak ṗ(t)dx(t), where dx(t) is the instantaneous increment of displacement, and A is a constant gain factor. A typical multimodal stiffness
stimulus as presented in the experiment is shown in Fig. 4.
2.2

Procedure and Adaptive Staircase Method

The experiment was designed to measure stiffness perception in one of four conditions, obtained by combining the grasp type (one or two-digits) with the feedback type(with or without vibration). We fit psychometric functions for stiffness perception separately from responses in each condition. In each condition, the experiment utilized anadaptive staircase procedure based on a two interval forced choice task, which required
participants to compare the comparison stimulus with variable stiffness k in the experimental condition concerned (i.e., with or without vibration feedback, one or two-digits)with a standard stimulus having a fixed reference value, k0 of stiffness and zero vibration feedback. In the vibration condition, the amplitude was A0 = 3.4. Participants felt

With Noise

4.89

2.04
0.81
0

5

10

15

20

25

Trial Number

30

35

40

Stiffness Coefficient (N/mm)

Yon Visell, Keerthi Adithya Duraikkannan, Vincent Hayward
Stiffness Coefficient (N/mm)

6Without Noise

4.89

2.04
0.81
0

5

10

15

20

25

Trial Number

30

35

40

Fig. 5. Example staircase pairs during a single adaptive procedure from the experimental set. The
conditions correspond to single-finger exploration. Left: With noise. Right: Without noise.

the comparison and standard stimuli in a two-alternative forced choice task in which
the two stimuli were presented in sequential random order. Participants responded via
keyboard entry indicating whether the first or second stimulus felt softer, or less stiff.
During each staircase, the comparison stiffness k was controlled using a 1-up, 1-down
adaptive procedure that converged toward the point of subjective equality (PSE), where
the comparison stimulus with stiffness k felt equally stiff to the standard stimulus withstiffness k0 . No vibration feedback was provided during standard stimulus presentation.
The value of k was manipulated during the experiment dependent on the responsesof the participant (Fig. 5). The initial value of k was set to a “high” or “low”, randomly
selected to lie between either 2k0 and 2.5k0 (“high” condition) or between k0 /2.5 andk0 /2 (“low” condition). When participants responded that the comparison felt stiffer(resp. less stiff), then the stiffness k was decreased (resp. increased) by one step unit.
The step size was initially large (12 dB) and was reduced to a smaller level (3 dB) after
five reversals in the direction of the threshold-tracking sequence. In each condition, twostaircase procedures were performed at a time, one beginning at “high” initial stiffness,and one at “low” stiffness. The two were performed in interleaved fashion, with one of
the two randomly selected for presentation on each trial. Each staircase was continued
until 20 reversals were reached. A total of eight staircases (four interleaved pairs) werecompleted by each subject, one pair in each of the experimental conditions. Subjects
were required to pause for 2 minutes between staircase pairs. The total experimental
duration for each subject was approximately 50 minutes.
2.3

Analysis of Experimental Data

The results of the experiment in each condition consisted of binary response data indicating whether or not the standard stimulus (no vibrations, and stiffness 2 N/mm), wasjudged less stiff than a comparison that varied in stiffness, with the latter controlled by
the adaptive procedure. Binary response data in each condition, i.e. at each vibrationlevel (A = 0, A0 ) and in each exploratory condition (one- or two-finger pressing) fromeach of the 8 subjects were fitted to a cumulative normal distribution using a probit
regression model. A total of 32 probit regression fits were performed; a representative
example is shown in Fig. 6. None of the fits failed to converge.
We investigated influences of vibration on stiffness perception by analyzing theeffect it had on the PSE, slope, and intercept of the psychometric functions during
either one- or two-finger squeezing. The analysis used a repeated measures analysis of variance (ANOVA) test. In the one-finger conditions, vibration feedback significantly increased PSE vs the non-vibration condition (mean 2.34 vs. 2.02 N/mm,
p < 0.05, F(1, 7) > 5.3), indicating that the stimulus felt softer when vibration wasVibrotactile Rendering of Volumetric Stiffness

7

Fig. 6. Psychometric curve fits from one participant. Left: Single-finger exploration, with andwithout vibration. Here, vibration elicited a bias in the PSE, or a reduction perceived stiffnessof the comparison stimulus, and a reduction in sensitivity (slope of the PSE). Right: Two-finger
exploration in the same conditions showed no significant effect (see text).

present. Despite this, data from two of eight participants evidenced a small negative effect (in both cases less than one standard deviation in size). Slope was not significantly
affected in the one-finger condition (p > 0.16, F(1, 7) = 2.4). However, the slope waslargest in the no-noise condition for all but two of the participants (the same two as
noted above). Thus, it appears possible that the lack of observed significance was due tooutliers or small sample size. In the two-finger condition, there was no significant effectof vibration on stiffness PSE (p > 0.3, F(1, 7) = 2.0) or slope (p > 0.75, F(1, 7) = 0.1),
indicating that two-finger stiffness perception was unaffected by the presence of vibration. The presence of vibration did not significantly affect intercept during interaction
with one- (p > 0.19, F(1, 7) > 2) or two-fingers (p > 0.8, F(1, 7) = 0.06), indicatingthat subjects were not biased to indiscriminately answer “softer” when vibration waspresent. Absent vibration, there was no significant difference between the PSE for one
and two finger squeezing (p > 0.15, F(1, 7) > 2.3), indicating a lack of effect of exploration type on stiffness judgements. However, participants were significantly moresensitive during two-finger squeezing, as evidenced by the higher slope (7.9 vs. 17.5,
p < 0.05, F(1, 7) > 5.8).

3Discussion and Conclusion: Rendering and Perception

In this paper, we presented a physically-motivated model for the perceptual integrationof action-phase-dependent vibration cues in softness perception during the palpationof a soft object. This motivates the hypothesis that an increase in vibration feedback
increases perceived softness, as has been observed in prior studies on pressing andwalking cited in the introduction. We used this model to derive an algorithm for themultimodal rendering of stiffness cues. We also presented a new high-fidelity haptic
interface designed to accurately reproduce both kinesthetic and vibrotactile cues during
one- or two-finger squeezing. To our knowledge, this is the first haptic force feedback
interface capable of reflecting nearly the entire dynamic range and frequency bandwidthof end-point forces that the haptic system is sensitive to.

8

Yon Visell, Keerthi Adithya Duraikkannan, Vincent Hayward

We further employed this device to assess the perceptual integration of stiffnessand vibration during one- and two-finger squeezing. The results support the notion that
action-phase dependent feedback supplies a cue that increases perceived object softness
during one finger squeezing. The experiment did not require training, and participants
were supplied no information related to the cues being provided. We can conclude thatvibration felt during single-finger pressing of a rigid surface is combined with the mechanical stiffness of the surface in the haptic perception of compliance. In the same
condition, it appeared that vibration might impair stiffness sensitivity, but our results
were inconclusive. There are a number of potential reasons this could be true, such as
sensory adaptation or cognitive factors, so we do intend to investigate further.
The results appear to be consistent with the mechanical model we proposed. Additionally, prior studies observed that displacement perception is highly accurate during
pinch grasping [9], and that stiffness perception by a single finger is poor when contact
area cues are not available [10]. Indeed, we observed two-finger stiffness perception to
be more accurate than one-finger squeezing. Further, based on our model, we predictthat vibration affects stiffness perception by altering displacement estimates (since theincremental amplitude of vibration is proportional to the increment of displacement).
Since two-finger size perception has been found to be highly accurate, one may expect
the brain to rely on this source when estimating stiffness [11].

References
1. Bensmaia, S., Hollins, M.: The vibrations of texture. Somatosens Mot Res 20(1) (2003)
33–43
2. Avanzini, F., Crosato, P.: Haptic-auditory rendering and perception of contact stiffness. In:
Haptic and audio interaction design. Springer (2006) 24–35
3. Asano, S., Okamoto, S., Matsuura, Y., Nagano, H., Yamada, Y.: Vibrotactile display approach that modifies roughness sensations of real textures. In: RO-MAN, 2012 IEEE, IEEE
(2012) 1001–1006
4. Visell, Y., Giordano, B.L., Millet, G., Cooperstock, J.R.: Vibration influences haptic perception of surface compliance during walking. PLoS one 6(3) (2011) e17697
5. Kildal, J.: 3d-press: haptic illusion of compliance when pressing on a rigid surface. In:
International Conference on Multimodal Interfaces and the Workshop on Machine Learningfor Multimodal Interaction, ACM (2010) 21
6. Kildal, J.: Kooboh: variable tangible properties in a handheld haptic-illusion box. In: Haptics: Perception, Devices, Mobility, and Communication. Springer (2012) 191–194
7. Wiertlewski, M., Lozada, J., Hayward, V.: The spatial spectrum of tangential skin displacement can encode tactual texture. Robotics, IEEE Transactions on 27(3) (2011) 461–472
8. Peng, G., Herrmann, H.J.: Density waves and 1/f density fluctuations in granular flow. Physical Review E 51(3) (1995) 1745
9. Durlach, N., Delhorne, L., Wong, A., Ko, W., Rabinowitz, W., Hollerbach, J.: Manual
discrimination and identification of length by the finger-span method. Perception & Psychophysics 46(1) (1989) 29–38
10. Srinivasan, M., LaMotte, R.: Tactual discrimination of softness. Journal of Neurophysiology
73(1) (1995) 88–101
11. Welch, R.B.: Meaning, attention, and the “unity assumption” in the intersensory bias of
spatial and temporal perceptions. Advances in psychology 129 (1999) 371–387