IEEE TRANSACTIONS ON HAPTICS,

VOL. 5,

NO. 1,JANUARY-MARCH 2012(c) 2012 IEEE., Personal use of this material is permitted. However,
permission to reprint/republish this material for advertising or
promotional purposes or for creating new collective works for resale orfor redistribution to servers or lists, or to reuse any copyrighted
component of this work in other works must be obtained from the IEEE.

85

Virtual Active Touch: Perception of Virtual
Gratings Wavelength throughPointing-Stick Interface
Shogo Okamoto, Member, IEEE, Takahiro Yamauchi, Masashi Konyo, Member, IEEE, andSatoshi Tadokoro, Fellow, IEEE
Abstract—Tactile feedback enhances the usability and enjoyment of human-computer interfaces. Many feedback techniques havebeen devised to present tactile stimuli corresponding to a user’s hand movements taking account of the concept of active touch.
However, hand movements may not necessarily be required for achieving natural tactile feedback. Here, we propose a virtual-activetouch method that achieves haptic perception without actual/direct hand movements. In this method, a cursor manipulated by a forceinput device is regarded as a virtual finger of the operator on the screen. Tactile feedback is provided to the operator in accordancewith cursor movements. To validate the translation of virtual roughness gratings, we compare the virtual-active-touch interface with aninterface that involves actual hand movements. By using the appropriate force-to-velocity gain for the pointing-stick interface, we showthat the virtual-active-touch method presents the surface wavelengths of the gratings, which is a fundamental property for texture
roughness, and that the gain significantly influences the textures experienced by the operators. Furthermore, we find that the perceived
wavelengths of objects scaled and viewed on a small screen are skewed. We conclude that although some unique problems remain tobe solved, we may be able to perceive the surface wavelengths solely with the intentions of active touch through virtual-active-touch
interfaces.
Index Terms—Isometric interface, vibrotactile texture, handheld device, force-to-velocity gain.

Ç
1

INTRODUCTIONCURRENTLY, many

human-computer interfaces such as
mobile phones and portable game players areequipped with tactile interfaces. Tactile feedback is expected to enhance the usability and enjoyment of theseinterfaces. Given that the importance of active hand
movements in haptic perception has been recognized [1],many tactile feedback technologies have thus far focused on
the generation of stimuli in response to the active hand
movements of their users. For example, for the presentationof virtual textures, tactile stimuli have been generated insynchronization with the position, velocity, and accelerationof a user’s finger or stylus (e.g., [2], [3], [4], [5]). In addition,
for many tactile feedback techniques such as in [6] and [7],the presentation of tactile stimuli in accordance with theuser’s active hand movements is considered to be a general
method for augmenting the validity of a technique.

. S. Okamoto is with the Department of Mechanical Science andEngineering, Nagoya University, Room 302, 2nd Building of School ofEngineering, Furo-cho, Chikusa-ku, Nagoya 464-0851, Japan.
E-mail: okamoto-shogo@mech.nagoya-u.ac.jp.
. T. Yamauchi is with the TORAY Industries, Inc., Tadokoro Laboratory,
Graduate School of Information Sciences, Tohoku University, 6-6-01
Aramaki Aza Aoba, Aoba-ku, Sendai 980-8578, Japan.
E-mail: yamauchi@rm.is.tohoku.ac.jp.
. M. Konyo and S. Tadokoro are with Tadokoro Laboratory, Graduate Schoolof Information Sciences, Tohoku University, 6-6-01 Aramaki Aza Aoba,
Aoba-ku, Sendai 980-8578, Japan.
E-mail: {konyo, tadokoro}@rm.is.tohoku.ac.jp.
Manuscript received 29 Sept. 2010; revised 12 July 2011; accepted 26 July
2011; published online 31 Aug. 2011.
Recommended for acceptance by K. MacLean.
For information on obtaining reprints of this article, please send e-mail to:
toh@computer.org, and reference IEEECS Log Number TH-2010-09-0064.
Digital Object Identifier no. 10.1109/ToH.2011.48.
1939-1412/12/$31.00 ß 2012 IEEE

1.1 Virtual Active Touch
Despite the widespread use of tactile feedback techniques,
our question is whether such hand movements are necessarily required for tactile interfaces. To address this question,
we proposed the possibility of virtual active touch—the
haptic perception when humans receive tactile feedback
without actual hand movements but with the intentions ofactive touch [8]. Although it does not involve direct hand
movements, virtual active touch may achieve haptic perception similar to that achieved by active touch. The present
study is based on our previous study [8] in which weinvestigated human perception for virtual active touch. In
the present study, we reached different conclusions with
Experiment II in Section 5 as a result of involving a large
number of participants.
Fig. 1 shows an example of a virtual-active-touch
interface. The user of a mobile phone manipulates a cursor
on the screen through a force-input device such as apointing stick or isometric joystick. The tactile stimulator isinstalled on the force-input device. The stimulator isactuated in response to cursor movement to provide tactile
feedback. The user receives the feedback in accordance withhis/her intentions of haptic exploration. As a result, she/he
may perceive the haptic properties of virtual objects or her/
his interaction with them just as well as when she/he
explores them with direct hand movements.
1.2

General Benefits of Virtual-Active-Touch
Interfaces
The drawbacks and advantages of the pointing stickcompared with others have been discussed [9], [10], [11],[12]. In general, the pointing stick does not exhibit faster
Published by the IEEE CS, RAS, & CES

86

IEEE TRANSACTIONS ON HAPTICS,

Fig. 1. Image of virtual-active-touch tactile interface.

positioning in a pointing task. It still has some benefits forcomputer interfaces lacking enough space for a touch pador for business people or software engineers that prefer tomaintain their wrist position for saving time. The pointing
stick provides another option for designers or users ofcomputer terminals. Furthermore, virtual-active-touch interfaces provide information terminals with some benefits
as follows.
To install tactile feedback function on ATMs or information kiosks whose screens are larger than those of personal
computers for the natural sense of virtual buttons or
textures, actuators with large output forces are requiredfor driving the entire touch screen in the case of vibrotactile
stimuli. In general, these large actuators have slow
responses and are unsuitable for generating complex tactile
stimuli. The virtual-active-touch interface can be located in
one place, and the mass that must be driven by thestimulator is relatively small. This allows the employmentof an actuator with quick responsiveness.
For handheld devices with small screens, the ratio of thescreen size to the finger is small. In the case of touch screens,the finger hides a large part of the display area when it scrolls
across the screen. For virtual-active-touch interfaces, thesensor for capturing the user’s force inputs and thestimulator can be placed outside of the screen (Fig. 1).
Hence, the user’s finger does not hide the display area.

1.3 Related Study
For natural tactile feedback, some researchers have alreadyused an indirect mapping between human input motions
and cursor movements on a computer. The devices
proposed by Poupyrev et al. [13] and Oakley et al. [14]fed back vibrotactile signals to their users tilting the devices
to select items in a menu. These devices may be included
among the virtual-active-touch interfaces in the broader
sense of indirect input motions for natural tactile feedback.
This study adopts a pointing stick to capture the user’s
virtual hand movements. There are few studies in theliterature on pointing sticks with tactile feedback function.
Campbell et al. introduced tactile feedbacks in a pointingstick interface. They showed that providing tactile feedback
could help navigation tasks [15] where a user of thepointing-stick draws a cursor on the screen along a circle
as quickly and accurately as possible. However, theirinterests were in the effects of tactile information on
navigation tasks rather than on perceptual properties. The
combination of a joystick, which has some similarity withthe pointing stick, and tactile feedbacks has been used forrealizing a Braille map system by Parente and Bishop [16].

VOL. 5,

NO. 1, JANUARY-MARCH 2012

Although a number of studies related to virtual-activetouch interfaces, the perceptual properties of textures have
yet to be studied for these interfaces.
Virtual active touch is neither active nor passive touch.
From the point of view that the user’s finger does notconsiderably move, it is similar to passive touch. On theother hand, the finger moves (albeit slightly) on the
pointing stick, and the user voluntarily exerts forces. Thisis considered similar to active touch. Differences inperceptual properties between active and passive touch
have been discussed from various views including roughness perception [17], [18], recognition of Braille letters or
figures [19], [20], and sensitivity enhancement [21]. Virtual
active touch may be a mixture of both modes or an entirely
different mode.

1.4 Motivation
The interest of this study is in whether or not the virtualactive-touch interface can deliver tactile perception of
textures. The perception of textures involves exploratory
hand movements; hence it is a suitable topic for studying
virtual active touch. In addition, texture presentation ispractically valuable for many purposes such as the deliveryof discrete icons or windows as different tactile textures, orthe supplementary notification of the moving distances of
the cursor while dragging. The presentation of texture
sensations for unique materials in the virtual world
naturally provides the users with enjoyment. For these
purposes, the fundamental knowledge is how accurately
the users of the virtual-active-touch interfaces perceive thevirtual textures. As a first step, we investigate theperception of roughness textures, for which the literature
is voluminous. We target the surface wavelengths ofroughness textures, which is one of the major parametersthat influence the perception of roughness textures such as
gratings. Given that the wavelength determines temporalfactors of vibrotactile stimuli depending on hand motions,
the wavelength has drawn the interest of many researchers
in order to examine relationships between roughness
perception and the relative movements of a finger pad
and textured surfaces [22], [23], [24], [25], [26], [27], [28].
1.5 Objective I (Experiment I)
In Experiment I, using a pointing-stick interface with avibrotactile stimulator, we show that the virtual-activetouch interface can provide users with information on
surface wavelengths. To this end, we compare the perceived
wavelengths represented by the pointing stick and those
represented by the linear-slider-type vibrotactile displaythat involves user’s hand movements. From informal
preliminary studies, we know that the force-to-velocity
gain of the input device is a crucial parameter. Therefore,
we also investigate the effects of this gain on the perceptionof wavelengths.
1.6 Objective II (Experiment II)
One of the merits of force-input interfaces such as apointing stick is the adaptability to variations in screen
size. Cursor movements are adjusted to the screen size bychanging the force-to-display gain. Objects on a screen are
visually scaled or enlarged so that they fit with the screen
size. It is expected that users naturally interact with scaled

OKAMOTO ET AL.: VIRTUAL ACTIVE TOUCH: PERCEPTION OF VIRTUAL GRATINGS WAVELENGTH THROUGH...

Fig. 2. Virtual object and cursors displayed to participants.

objects on a small screen. However, no study has reported
the tactile perception of these scaled objects. We examine
the human perceptual properties of the wavelengths byvarying the visual scales of roughness textures displayed ona screen.

2

VISUAL AND TACTILE STIMULI

2.1 Tactile Stimuli
2.1.1 Virtual Gratings
Virtual sinusoidal gratings are used as tactile textures.
When a user’s finger traverses the virtual grating, sinusoidal displacements are applied to his/her finger pad. The
profile of the displacements applied to the finger skin isexpressed as
 


xðtÞ
þ1 ;
ð1Þ
yðtÞ ¼ A sin 2

where xðtÞ and  are the position of the finger or cursor
along the virtual texture and the spatial surface wavelengthof the texture, respectively. These sinusoidal displacements
have been used by some researchers to investigate theperception of virtual gratings presented by haptic displays[29], [30], [31]. A vibratory actuator is used as the tactile
stimulator; the voltage supply to the actuator is expressedby (1) where A is the amplitude of the voltage. We fix A andvary . In association with hand movements,  influences
texture perception. When the instantaneous velocity of thefinger is xðtÞ,_the frequency of the vibratory stimuli
presented to the user is given by fðtÞ ¼ xðtÞ=;_this affects
the perception of the textures. On the other hand, A affects
the perception independently on the hand movements. We
fix A because the interest of this study is in the perception
caused by virtual movements of the hand.

2.1.2 Stimulator
The vibratory actuators are piezo-stack actuators(AHB800C801FPOLF, NEC/TOKIN, Sendai, Japan). Theiroutput displacement is approximately linear to the applied
voltage. The output displacements are 34.7 and 63.0 m for
A values of 30 and 50 V, respectively (these voltages are usedin the latter experiments). The amplitude-frequency response of the vibrator is almost flat in the range used for the87

Fig. 3. Pointing-stick-type (PS-type) tactile interface.

experiments. The displacement reaches 3 dB at frequencies
of 275 and 195 Hz for A values of 30 and 50 V, respectively. A
bipolar amplifier (BA4825, NF, Yokohama, Japan) whose
nominal frequency response is up to 2 MHz is used.
The control frequency for updating the supplied voltage
is 5 kHz. It is controlled by a personal computer with ARTLinux through an IO board (LPC-361216, Interface corp.,
Hiroshima, Japan). The output displacement is not feedback controlled. This is because the force of a humanfinger, being much smaller than the output force of thestimulator (approximately 800 N) does not degrade theoutput displacement.
The output displacements are sufficient to allow participants to experience tactile sensations while exploring
virtual textures. In informal introspective reports, theparticipants agreed that through the tactile display systems
mentioned later, they experience sensations similar to thoseacquired when they explore rough textures such as
sandpapers or gratings using a stylus with a blunt point.

2.2 Visual Stimuli
The image of the target texture used as the visual stimulus
should be a familiar item such that every participant can
imagine its actual dimensions. We use the image of a brownbrick, as shown in Fig. 2. Its dimensions as observed on acomputer screen are 120  60 mm, which are close to thosecommonly used in Japan. The cursor that is manipulated by
the interface is a 6-mm square. Even when this is scaled
down in Experiment II, it is still sufficiently large for theparticipants to see without stress. Its movements are
confined along the X-axis.

3

EXPERIMENTAL EQUIPMENT

Two types of tactile interfaces are prepared. One is a pointingstick-type (PS-type) interface for virtual active touch. The
other is a linear-slider-type (LS-type) interface for tactile
exploration with hand movements. Participants compare thegratings presented through these two types of interfaces.

3.1 Pointing-Stick-Type (PS-Type) Tactile Interface
The PS-type tactile interface is shown in Fig. 3. When the user
applied tangential forces with a finger along the X-axis, thecursor on a computer screen moved according to the applied
force. When the cursor slid on the brick, vibratory stimuli

88IEEE TRANSACTIONS ON HAPTICS,

VOL. 5,

NO. 1, JANUARY-MARCH 2012

interfaces. The participants adjust the surface wavelength of
the gratings () presented by the PS-type interface so that it
feels similar or identical to that presented by the LS-type
interface, following the psychophysical method of adjustment. The control gain ( in (2)) is varied in order to examine
its effect on texture sensations.

4.1

Fig. 4. Linear-slider-type (LS-type) tactile interface.

were presented to the participants through a tactile stimulator. A contact shoe was installed on top of the vibrator tosecure an adequate contact area between the finger and thestimulator. The shoe was circular in shape with a diameter of
20 mm. A six-axis force sensor (MINI2/10, BL-AUTOTECH,
Kobe, Japan) with the resolution of 1:96  102 N and a 100Hz-low-pass filter was installed beneath the stimulator tomeasure the tangential force applied to the stimulator along
the X-axis. The IO board on the computer received thesensory information and output the voltage to the stimulator
through the amplifier.
We employed a linear transformation between the_applied force (F ðtÞ) and the cursor velocity (xðtÞ)to simplify
the analysis. The equation to transform these two types of
information is expressed as_ ¼ F ðtÞ;
xðtÞ

ð2Þ

where  is the control gain. The control frequency of thissystem was 5 kHz, i.e., the latency between the motion
input and vibrotactile output was less than 200 s.

3.2 Linear-Slider-Type (LS-Type) Tactile Interface
The LS-type tactile interface is shown in Fig. 4. The
vibratory stimulator was installed on top of the linear slider
and was moved along the X-axis by the participant’s hand
movement. When the operator placed a finger on thevibrator and moved his/her hand along with the linear
slider, the cursor on the computer screen moved while
tactile stimuli were presented to the participant’s finger.
The movable length on the guide was approximately
200 mm. The position of the stimulator on the guide wasmeasured by an optical encoder (SR-P1000, Canon, Tokyo,
Japan) whose spatial resolution was 0.4 m. The encoder
pulses were counted by a counter board (LPC-632104,
Interface corp., Hiroshima, Japan) installed on the computer. Its IO board then output the voltage to the stimulator.
The velocity of the cursor controlled by the LS-type
interface was equal to the actual hand velocity measured
by the encoder, which means the control gain was 1.

4

EXPERIMENT I: CONVEYANCE OF SURFACEWAVELENGTHS BY VIRTUAL ACTIVE TOUCH

In this experiment, the participants compare the virtual
gratings they perceive through both the PS-type and LS-type

Experimental Tasks, Stimuli, and Participants

4.1.1 Tasks
The reference and test stimuli were presented by the LS- andPS-type tactile interfaces, respectively. The participants
adjusted  of the test stimulus so that the gratings presented
by the PS-type interface became close to those obtained by theLS-type interface. They could adjust  only for the PS-type
interface by pressing keys on a keyboard. A single press of a
key changed  by 0.05 mm. Random values were assigned tothe initial values for the test . The participants could freely
switch between the PS- and LS-type interfaces that wereplaced on a desk before them. The duration of a single trial
was limited to 30 s in order to prevent unexpectedly longdeliberation by the participants. If the participant did notcomplete a trial within 30 s, the trial was repeated after thecompletion of all remaining trials. These retrials happened atmost a few times for individual participants.
4.1.2 Stimuli
Eight reference stimuli whose  values varied from 0.4 to
1.8 mm in 0.2 mm steps were used in random order. The four values employed were: 1:02  102 , 2:04  102 , 4:08  102 ,and 6:12  102 mm/sN. Four trials were conducted for eachcondition. In total, 128 (8 reference stimuli  4 gains  4
repetitions) trials were conducted for each participant.
As a result of a limitation in the voltage amplifiers used
for the vibrators, the maximum applied voltage (defined by
A in (1)) was set to 30 and 50 V when the reference
wavelengths were 0.4-0.8 mm and 1.0-1.8 mm, respectively.
The same A value was used for both the reference and test
stimuli in a single trial.
4.1.3 Participants
Five voluntary students from the authors’ laboratory(excluding the authors) in their 20s, given informed consent,
performed the tasks. They wore earplugs and headphones
through which pink noise was audible to prevent sounds
generated by the vibrator from being heard. Before performing the experiment, the participants practiced manipulating
the cursors using both the PS- and LS-type tactile interfaces.
During the experiments, they were instructed to match thevelocities and strokes of the cursor movements for the PStype interface with those of the LS-type interface. None ofthem voluntarily reported feeling that their exploratory
movements were unnatural and that this matching of velocity
prevented them from conducting the tasks. No reference
marker to instruct/teach the movements of the cursor to theparticipants was presented. Individuals could move thecursor at speeds and rhythms with which they felt comfortable. The participants could practice until they werecomfortable with the tasks, which usually took a few minutes.
4.2 Experimental Results
Fig. 5 shows the relationships between the adjusted  andthe reference  for each  value. The error bars indicate

OKAMOTO ET AL.: VIRTUAL ACTIVE TOUCH: PERCEPTION OF VIRTUAL GRATINGS WAVELENGTH THROUGH...

89

Fig. 6. Adjusted wavelength versus reference wavelength during
application of optimal gain (0 ).
Fig. 5. Adjusted wavelength versus reference wavelength by control
gain ().

standard deviations across the participants. The dashed line
denotes the relationship in which the adjusted andreference  values are identical. It can be observed fromthe figure that the adjusted  increased with the reference 
for all  values. Also, the adjusted  differed for different
 values. Analysis of covariance was applied to theexperimental results, with the reference  as the covariant
and  as the group factor. The results of the analysis
showed that there was a significant difference between theslopes of the regression lines at different values of (F ð3; 152Þ ¼ 14:09, p ¼ 3:78  108 ). At smaller values of ,
the adjusted  values tended to be smaller than those forlarger gains. This trend was observed for all participants.

4.3 Optimal Gain
From the experimental data above, we estimate the optimal
gain 0 , defined as the gain at which the adjusted  is equal
to the reference . The value of 0 is estimated by
performing a regression analysis of the experimental
results. From this analysis, the adjusted wavelength (a ) is
given bya ¼ 5:41  104  þ 1:05r  0:152;
where r is the reference wavelength. 0 is defined as
Z 1:8
ða  r Þ2 dr :0 ¼ arg minð3Þ

movements of the cursors between the PS- and LS-type
interfaces during the experiments.
Fig. 7 shows the relationship between  and the averagecursor velocity for the two types of interfaces. The error bars
indicate the standard deviations across the participants. The
average velocities were calculated for values greater than
5 mm/s. A significant positive correlation between the
cursor velocity and  was observed for the PS-type interface(r ¼ 0:58; t0 ð18Þ ¼ 3:84; p ¼ 6:0  104 ). On the other hand,no such correlation was observed for the LS-type interface(r ¼ 0:17; t0 ð18Þ ¼ 0:81; p ¼ 0:22). Although the participants
were instructed to balance the cursor movements for bothtypes of interface, the cursor velocities of the two types
differed when  was large.
Based on the observed differences in cursor velocities
between the two types of interfaces, we explain the influenceof  on the judgment of  as follows: the participants aresupposed to have judged  values based on the relationshipbetween the cursor movement and the corresponding
vibrotactile frequency. This relationship is given by  ¼__ affects the judgment of .
fðtÞ  xðtÞ,which indicates that xðtÞWhen the cursor velocity for the PS-type interface was larger
than that for the LS-type interface, participants felt the  forthe PS-type interface to be larger than that for the LS-type
interface. Hence, differences in the cursor velocities between
the two types of interfaces might have led to differences in the
judgment of .

ð4Þ

0:4

From this equation, we obtain 0 ¼ 174 mm/sN. The task
described in Section 4.1 was then performed by the same
participants with this value of 0 . The results are shown in
Fig. 6. The adjusted  values were almost same as thereference  values displayed by the LS-type tactile interface(R2 ¼ 0:995). It is concluded that the PS-type tactile interface with a tuned gain can display the surface wavelengths
approximating those displayed by the LS-type tactileinterface.

4.4

Discussion: Effects of Control Gain on Adjusted
Wavelength
We analyzed the movements of the cursors and found thatthe cursor velocity varied with . It should be noted that theparticipants were instructed to equalize the apparent

Fig. 7. Control gain versus average cursor velocity.

90

IEEE TRANSACTIONS ON HAPTICS,

VOL. 5,

NO. 1, JANUARY-MARCH 2012

Fig. 8. Visually scaled virtual object. Left: fundamental scale size (reference/test stimulus). Center: half-scale size (test stimulus). Right: Quarterscale size (test stimulus).

5

EXPERIMENT II: TEXTURE PRESENTATIONS FORSCALED OBJECTS

Large-sized objects need to be scaled so that they arerecognizable on small screens such as those of mobile
terminals. In this section, the perception of  for scaled
textures is experimentally investigated.

5.1 Method for Displaying Scaled Gratings
To present virtual gratings independent of their scaling on
the screen, the velocity of the cursor and the spatial
characteristics of the gratings are scaled at a scaling rateequal to that of the visual image. The participants are
assumed to judge  from the relationship between the
cursor velocity and the vibrotactile feedback presented. This
relationship, which is expressed in the form of a definition_of the vibrotactile frequency, fðtÞ ¼ xðtÞ=,should be
maintained before and after the objects are scaled. In order_to maintain the vibratory frequencies, xðtÞand  are scaled
at the same rate as are the visual dimensions of the object on_ and  are scaled at the same rate,
the screen. As long as xðtÞfðtÞ is unaffected.
5.2 Experimental Conditions and Tasks
Using only the PS-type interface, the participants compared
the gratings of scaled and nonscaled objects and matched
their  by the psychophysical method of adjustment. The
reference stimulus was a nonscaled object. The test stimulus_
was an object whose visual dimensions, , and xðtÞwere
scaled at the same rate. The participants adjusted the  ofthe test stimulus so that they felt the textures of the scaled
and nonscaled objects were similar or identical.
5.2.1 Reference and Test Stimuli
The reference stimulus, which was the nonscaled object, isshown in Fig. 8 left. The image was the same as the oneused in Experiment I. The dimensions of the object, as
observed on the screen, were 120  60 mm. The five
wavelengths prepared for the reference stimulus varied
from 0.6 to 1.4 mm in 0.2 mm steps. The voltage amplitude
applied to the tactile stimulator was 30 V. The control gain() was set to the optimal value obtained in Experiment I:
 ¼ 1:74  102 mm/sN.
The three scaling values used for the test stimuli were0.25, 0.5, and 1.0 (nonscaled). The visual stimuli for thesescaled objects are shown in Fig. 8. In addition to the bricks,
the cursor was scaled at the same rate. This maintains thedimensional relationships in the virtual world, which islikely to allow the participants to feel gratings scaled bydifferent factors as similar. Three trials were conducted foreach of the five reference wavelengths and three scaling
values. In total, 45 (five reference wavelengths  3 scaling
values  3 repetitions) trials were conducted for eachparticipant. The order in which the stimuli were displayed
was random.

5.2.2 Tasks
The participants explored the reference and test stimuli inalternation, switching freely between the two types ofstimuli by pressing a key on a keyboard. They adjusted the of the test stimuli by pressing other keys. The duration ofa single trial was limited to 30 s. If the participants did notcomplete the trial within that time, the trial was repeatedafter the completion of the remaining trials. They practiced
the abovementioned tasks before performing the experiments until they were comfortable with the tasks. Seventeen
voluntary participants performed these tasks.
5.3 Experimental Results
Fig. 9 shows averages of the adjusted  for each scaling value.
The error bars indicate the standard deviations acrossthe participants. Analysis of covariance was applied to theresults with the reference  as the covariant and the scaling
size as the group factor. The analysis showed a significant
effect of scaling size (F ð2; 251Þ ¼ 3:84, P ¼ 0:023), indicatingthat the reported  values increased as the scaling size

Fig. 9. Adjusted wavelength versus reference wavelength at different
scaling values.

OKAMOTO ET AL.: VIRTUAL ACTIVE TOUCH: PERCEPTION OF VIRTUAL GRATINGS WAVELENGTH THROUGH...

decreased. The participants perceived the  of scaled objects
as different from that of nonscaled objects.
From the perspective of psychophysics, the effect of thescaling size is nonnegligible. The reported values werelarger than the reference values by 9.3 percent on averagewhen the scale size was 0.25. The 75 percent limen of real
gratings with a surface wavelength of 1.0 mm is approximately 5-8 percent [24], [18]. The limen for virtual gratings
is possibly higher than those for real gratings [30]. Further,because the participants judged the wavelengths based on
the vibrotactile frequencies, we compare the distortion ofperceived wavelengths with the limen of the frequencies
that are approximately 20-30 percent [32], [33]. However,this value is not also directly comparable with theexperimental results because the participant’s judgment
was based not only on the vibratory frequencies but also on
the cursor velocities. If we take a conservative comparisonin the absence of directly comparable data, the effect of thescaling value on the perceived wavelengths may be as large
as the just noticeable difference.
It is concluded that even when the scaling ratios of visualobjects, cursor movements, and surface wavelengths ofgratings are identical and the relationships between the
cursor movements and the corresponding tactile feedback
stimuli are maintained, a scaling ratio of one-fourth skewed
the perceived wavelengths by 9.3 percent on average. Thus,
visually scaled textures do not provide the correct texture
sensations. Of course, the skew of perceived wavelengths
can be corrected by adjusting the force-to-velocity gain of
the pointing stick device. Furthermore, a 10 percent
distortion may not be that significant for many applications,especially for entertainment.

_ ¼   fðtÞ before and after scaling. Howrelationship xðtÞever, the perceived wavelengths were skewed when they
were scaled. One potential cause underlying the skewed
perception is a nonlinear perception of velocity. When
Stevens’ power law holds for the perception of visualvelocity of a moving object, the perceived surface wavelength is described asm_^ ¼ kxðtÞ
;
ðtÞ
fðtÞ

DISCUSSIONS

6.1

Can Virtual Active Touch Present Surface
Wavelengths of Virtual Gratings?
As shown in Experiment I (Section 4.3), the participants
perceived the  of virtual gratings through the virtualactive-touch interface as well as they did through the LStype interface. However, it was found that the force-tovelocity gain of the pointing stick significantly influences
the perceived wavelengths. The proper gain may depend on
the types of input devices. For example, gains need to be
specifically tuned for touch pads or joysticks. Further, weneed to elucidate related issues including the identificationof factors that affect the optimal gain, or the perception
under a nonlinear control gain.
As to the scaling effects of the gratings, when thegratings and cursor velocities were scaled at the same ratio,
the participants reported  values that were larger than
expected. The difference between the reported and expected was approximately 10 percent on average at a scaling
factor of 0.25. This difference may be as large as thediscrimination threshold of the surface wavelengths. The
virtual-active-touch interface cannot present surface wavelengths when they are scaled.
6.2

Why Was Perceived  Skewed When Gratings
Were Scaled?
In Experiment II, the perceived wavelength was investigated when the gratings were scaled while maintaining the

ð5Þ

where k and m are constants. Algom and Cohen-Raz
summarized the power indices (m) reported by several
research groups [34]. Although the reported m values
depend on methods of investigation, the majority are lessthan 1. When m is smaller than 1, the decrease in theperceived velocity is smaller than the actual decrease. As aresult, ^ becomes larger than the actual . For example,
when the texture is scaled by a factor of 0.25, the cursor
velocity also becomes one-fourth. However, since theperceived velocity does not decrease as much, the perceived
 becomes larger than the actual .
Another possible reason is that the optimal gain depends
on the scaling factor. In Experiment II, the same optimal
gain acquired in Experiment I was used without respect toscaling rates. This gain might not be optimal for the scaling
rates of 0.5 and 0.25, and thus cause the distortion ofperceived wavelengths. In addition, the visual shrinkage ofthe bricks might have influenced the perceived wavelengths. Investigating the reasons for this distortion is thenext challenge.

7
6

91

CONCLUSIONS

Virtual active touch is the perception of touch and its
underlying process when a person touches objects on acomputer screen through an input interface that does not
involve direct exploratory hand movements. The objectiveof this study was to validate the capabilities of virtualactive-touch interfaces to present users with the surface
wavelength, which fundamentally specifies the gratings.
Furthermore, the effects of force-to-velocity gains on theperceived wavelength and the presentation of visuallyscaled gratings were investigated.
For validation, we compared the perceived wavelengths
by an isometric tactile interface (pointing-stick-type) for
which the hand moved only slightly, and an isotonic
interface (linear-slider-type) for which it moved as much as
the cursor. The results of psychophysical experiments
confirmed that the pointing-stick-type tactile interface
presented wavelengths similar to those presented by thelinear-slider-type tactile interface with a tuned force-tovelocity gain. Furthermore, it was found that this gainsignificantly affected the perceived wavelengths. For visually scaled gratings, the virtual-active-touch interface
skewed the perceived wavelengths. When the scaling factor
was one-fourth, a distortion of approximately 10 percent wasobserved in the wavelengths perceived by the participants.
Certain problems remain to be solved. These include thespecification of factors that influence the optimal control
gain, as well as the validation of virtual active touch forperceived roughness or other vibrotactile textures given

92IEEE TRANSACTIONS ON HAPTICS,

that the present study verified only the perception of
wavelengths. Collectively, the findings of this study suggestthat virtual-active-touch interfaces are capable of presenting
the surface wavelengths of gratings at least when the gain iscarefully tuned and scaling is not applied to the virtual
gratings. With further investigations, virtual active touch is
expected to enhance the usability and enjoyment of tactile
feedback functions on mobile or portable game players.

ACKNOWLEDGMENTS
This work was supported in part by MIC SCOPE(082102006).

REFERENCES
[1]
[2]
[3]
[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]
[13]

[14]
[15]

[16]

J.J. Gibson, “Observations on Active Touch,” Psychological Rev.,
vol. 69, pp. 477-490, 1962.
Y. Ikei, K. Wakamatsu, and S. Fukuda, “Vibratory Tactile Displayof Image-Based Textures,” IEEE Computer Graphics and Applications, vol. 17, no. 6, pp. 53-61, Nov./Dec. 1997.
A.M. Okamura, J.T. Dennerlein, and R.D. Howe, “Vibration
Feedback Models for Virtual Environments,” Proc. IEEE Int’l Conf.
Robotics and Automation, pp. 674-679, 1998.
D.G. Caldwell, N. Tsagarakis, and C. Giesler, “An Integrated
Tactile/Shear Feedback Array for Stimulation of Finger Mechanoreceptor,” Proc. IEEE Int’l Conf. Robotics and Automation, pp. 287292, 1999.
M. Konyo, A. Yoshida, S. Tadokoro, and N. Saiwaki, “A Tactile
Synthesis Method Using Multiple Frequency Vibration for
Representing Virtual Touch,” Proc. IEEE/RSJ Int’l Conf. Intelligent
Robots and Systems (IROS), pp. 3965-3971, 2005.
J. Pasquero, J. Luk, V. Levesque, Q. Wang, V. Hayward, and K.E.
MacLean, “Haptically Enabled Handheld Information Display
with Distributed Tactile Transducer,” IEEE Trans. Multimedia,
vol. 9, no. 4, pp. 746-753, June 2007.
L. Winfield, J. Glassmire, E.J. Colgate, and M. Peshkin, “T-Pad:
Tactile Pattern Display through Variable Friction Reduction,”
Proc. World Haptics Second Joint Eurohaptics Conf. and Symp. Haptic
Interfaces for Virtual Environment and Teleoperator Systems, pp. 421426, 2007.
T. Yamauchi, M. Konyo, S. Okamoto, and S. Tadokoro, “Virtual
Active Touch: Perceived Roughness through a Pointing-StickType Tactile Interface,” Proc. Third Joint Eurohaptics Conf. andSymp. Haptic Interfaces for Virtual Environment and Teleoperator
Systems, pp. 605-610, 2009.
S.K. Card, W.K. English, and B.J. Burr, “Evaluation of Mouse,
Rate-Controlled Isometric Joystick, Step Keys, and Text Keys for
Text Selection on a CRT,” Ergonomics, vol. 21, no. 8, pp. 601-613,
1978.
I.S. MacKenzie, A. Sellen, and W. Buxton, “A Comparison of Input
Devices in Elemental Pointing and Dragging Tasks,” Proc. ACM
Conf. Human Factors in Computing Systems: Reaching throughTechnology (CHI ’91), pp. 161-166, 1991.
A.K. Mithal and S.A. Douglas, “Differences in Movement
Microstructure of the Mouse and the Finger-Controlled Isometric
Joystick,” Proc. SIGCHI Conf. Human Factors in Computing Systems(CHI ’96), pp. 300-307, 1996.
C. Sutter and M. Ziefle, “Interacting with Notebook Input Devices:
An Analysis of Motor Performance and User’s Expertise,” Human
Factors, vol. 47, no. 1, pp. 169-187, 2005.
I. Poupyrev, S. Maruyama, and J. Rekimoto, “Ambient Touch:
Designing Tactile Interface for Handheld Device,” Proc. 15th Ann.
ACM Symp. User Interface Software and Technology (UIST ’02),
pp. 51-60, 2002.
I. Oakley, J. Ängeslevä, S. Hughes, and S. O’Modhrain, “Tilt andFeel: Scrolling with Vibrotactile Display,” Proc. EuroHaptics,
pp. 316-323, 2004.
C.S. Campbell, S. Zhai, K.W. May, and P.P. Maglio, “What You
Feel Must Be What You See: Adding Tactile Feedback to theTrackpoint,” Proc. Seventh IFIP Conf. Human-Computer Interaction,
pp. 383-390, 1999.
P. Parente and G. Bishop, “Bats: The Blind Audio Tactile Mapping
System,” Proc. ACM South East Conf., 2003.

VOL. 5,

NO. 1, JANUARY-MARCH 2012[17] S.J. Lederman, “The Perception of Surface Roughness by Active
and Passive Touch,” Bull. Psychonomic Soc., vol. 18, no. 5, pp. 253255, 1981.
[18] G.D. Lamb, “Tactile Discrimination of Textured Surfaces: Psychophysical Performance Measurements in Humans,” J. Physiology,
vol. 338, pp. 551-565, 1983.
[19] M.A. Heller, G.J. Rogers, and C.L. Perry, “Tactile Pattern
Recognition with the Optacon: Superior Performance with Active
Touch and the Left Hand,” Neuropsychologia, vol. 28, no. 9,
pp. 1003-1006, 1990.
[20] F. Vega-Bermudez, K.O. Johnson, and S.S. Hsiao, “Human Tactile
Pattern Recognition: Active versus Passive Touch, Velocity Effects
and Patterns of Confusion,” J. Neurophysiology, vol. 65, no. 3,
pp. 531-546, 1991.
[21] R. Helminen, H. Mansikka, and A. Pertovaara, “Lowered or
Increased Cutaneous Sensitivity during Movement Depends on
Stimulus Intensity,” Perceptual and Motor Skills, vol. 78, no. 3,
pp. 721-722, 1994.
[22] S.J. Lederman, “Tactual Roughness Perception: Spatial andTemporal Determinants,” Canadian J. Psychology, vol. 37, no. 4,
pp. 498-511, 1983.
[23] A.W. Goodwin, K.T. John, K. Sathian, and I. Darian-Smith,“Spatial and Temporal Factors Determining Afferent Fiber
Responses to a Grating Moving Sinusoidally over the Monkey’s
Fingerpad,” The J. Neuroscience, vol. 9, no. 4, pp. 1280-1293, 1989.
[24] J.W. Morley, A.W. Goodwin, and I. Darian-Smith, “Tactile
Discrimination of Gratings,” Experimental Brain Research, vol. 49,
pp. 291-299, 1983.
[25] M. Hollins and S.R. Risner, “Evidence for the Duplex Theory of
Tactile Texture Perception,” Perception & Psychophysics, vol. 62,
no. 4, pp. 695-705, 2000.
[26] E.M. Meftah, L. Belingard, and C.E. Chapman, “Relative Effects of
the Spatial and Temporal Characteristics of Scanned Surfaces on
Human Perception of Tactile Roughness Using Passive Touch,”Experimental Brain Research, vol. 132, pp. 351-361, 2000.
[27] E. Gamzu and E. Ahissar, “Importance of Temporal Cues for
Tactile Spatial-Frequency Discrimination,” The J. Neuroscience,
vol. 21, no. 18, pp. 7416-7427, 2001.
[28] C.J. Cascio and K. Sathian, “Temporal Cues Contribute to Tactile
Perception of Roughness,” The J. Neuroscience, vol. 21, no. 14,
pp. 5289-5296, 2001.
[29] A.M. West and M.R. Cutkosky, “Detection of Real and Virtual
Fine Surface Features with a Haptic Interface and Stylus,” Proc.
ASME IMECE Sixth Ann. Symp. Haptic Interfaces, pp. 1-8, 1997.
[30] H.Z. Tan, B.D. Adelstein, R. Traylor, M. Kocsis, and E.D.
Hirleman, “Discrimination of Real and Virtual High-Definition
Textured Surfaces,” Proc. Symp. Haptic Interfaces for Virtual
Environment and Teleoperator Systems 2006, pp. 3-9, 2006.
[31] S.A. Cholewiak, K. Kim, H.Z. Tan, and B.D. Adelstein, “A
Frequency-Domain Analysis of Haptic Gratings,” IEEE Trans.
Haptics, vol. 3, no. 1, pp. 3-14, Jan.-Mar. 2010.
[32] G.D. Goff, “Differential Discrimination of Frequency of Cutaneous
Mechanical Vibration,” J. Experimental Psychology, vol. 74, no. 2,
pp. 294-299, 1967.
[33] M. Rothenberg, R.T. Verrillo, S.A. Zahorian, M.L. Brachman, and
S.J.B. Jr, “Vibrotactile Frequency for Encoding a Speech Parameter,” J. Acoustical Soc. Am., vol. 62, no. 4, pp. 1003-1012, 1977.
[34] D. Algom and L. Cohen-Raz, “Visual Velocity Input-output
Functions: The Integration of Distance and Duration onto
Subjective Velocity,” J. Experimental Psychology: Human Perception
and Performance, vol. 10, no. 4, pp. 486-501, 1984.
Shogo Okamoto received the BS degree inengineering from Kobe University in 2005 andthe MS and PhD degrees in information
sciences in 2007 and 2010 from Graduate
School of Information Sciences, Tohoku University, respectively. Since 2010, he has been
an assistant professor at the Graduate School ofEngineering, Nagoya University. He received the
ROBOMEC Award in 2007, 2008, and 2009 from
Robotics and Mechatronics division of the
JSME, the IEEE RAS Japan Chapter Young Award (ICRA ’08) in
2008, the Award for the Best Hands on Demo at EuroHaptics ’08, etc.
He is a member of the IEEE.

OKAMOTO ET AL.: VIRTUAL ACTIVE TOUCH: PERCEPTION OF VIRTUAL GRATINGS WAVELENGTH THROUGH...

Takahiro Yamauchi received the BS and MS
degrees from School of Engineering, and Graduate School of Information Sciences, Tohoku
University, in 2008 and 2010, respectively. Since
2010, he has been working for TORAY Industries, Inc. He researched on tactile telepresence
systems and tactile interfaces for mobiles. His
research interests include vibrotactile interfaces.

Masashi Konyo received the BS, MS, and PhD
degrees in engineering from Kobe University in
1999, 2001, and 2004, respectively. Since 2009,he has been an associate professor in the
Graduate School of Information Sciences, Tohoku University. His current research interests
include haptic perception, tactile interfaces, andhuman cooperative robotics. He received theBest Poster Award at the WorldHaptics 2007,
the Best Hands on Demo Award at the EuroHaptics ’08, the Best Paper Award of the Journal of Robotics andMechatronics in 2010, etc. He is a member of the IEEE.

93

Satoshi Tadokoro (M’89-SM’06-F’09) received
the ME degree in 1984 from the University ofTokyo, and the DE degree in 1991 from Kobe
University. He has been a professor in the
Graduate School of Information Sciences,
Tohoku University, since 2005. He was the
project leader of the MEXT DDT Project on
rescue robotics in 2002-2007 which included
contributions from more than 100 professors
nationwide. He established RoboCupRescue in
1999, the TC on Rescue Engineering of SICE in 2000 (the first chair),IEEE Robotics and Automation Society TC on Safety, Security andRescue Robotics in 2001 (the first chair), and the International
Rescue System Institute (IRS) in 2002. He is at present the presidentof IRS, the trustee of The RoboCup Federation, the chair of JSME
Robotics Mechatronics Division, and a IEEE Robotics and Automation
Society AdCom member. He is a fellow of the IEEE.

. For more information on this or any other computing topic,
please visit our Digital Library at www.computer.org/publications/dlib.