A novel multimodal tactile module that canprovide vibro-thermal feedback
Masashi Nakatani1,4,5 , Katsunari Sato2,5 , Kunio Sato3,5 , Yuzuru Kawana3,5 ,Daisuke Takai3 , Kouta Minamizawa4 and Susumu Tachi1
1Institute of Gerontology, University of Tokyo{nakatani, tachi}@iog.u-tokyo.ac.jp
2
Nara Women’s University
katsu-sato@cc.nara-wu.ac.jp
3
Alps Electric Co. Ltd{kunio.sato, yuzuru.kawana, daisuke.takai}@jp.alps.com
4Graduate School of Media Design, Keio University
kouta@kmd.keio.ac.jp
5These authors equally contributed to this studyAbstract. This paper describes a novel tactile module that can provide
both vibratory and thermal feedback onto the skin. The module is composed of two different components; miniaturized vibrator and four unitsof Peltier devices. The dimension of the vibro-thermal tactile feedback
device is 16 × 32 × 7 mm, which is small enough for attaching to fingerpad. We describe basic concept of this module, and the state of current
prototype including possible applications.
Keywords: Tactile Display · Thermal Feedback · Vibrotactile Feedback· Wearable device

1IntroductionThere are enormous numbers of tactile displays proposed so far. Pin-based tactile
display is most studied in analogous to visual displays. This type of tactile display uses multiple contractors that are driven with different actuators [8]. Othertactile displays utilize physical phenomena to induce skin deformation such as
electrostatics [14], surface elastic wave [10] or to directly activate sensory afferent
by electrocutaneous stimuli [7].
In addition to providing mechanical tactile feedback to provoke tactile sensation, thermal displays also attract attentions recent years [6]. They used water
circulation [1], Peltier devices [2], or heaters [5]. The thermal display can not only
display thermal cues, but they can also produce tactile perception of material
properties [6].
It would be beneficial for the research community to have multimodal tactile displays that can present vibratory and thermal tactile cues simultaneously.
Based on recent progress of psychology on vibrotactile and thermaltactile perceptions, it is expected that multimodal tactile display may enhance realistic

2M. Nakatani et al.

experience while interacting with presented object. Although multimodal tactile sensors have been proposed [9] and commercially available [4], only a few
multimodal tactile displays have been proposed [3]. Considering mobile or wearable computing purposes in the usage of multimodal tactile display, the size of
hardware should be taken into account.
Here we developed a small-size tactile module that can provide both vibratory and thermal tactile feedback onto the skin, which is small enough forwearable use. We also evaluated our developed device, and further discussed
possible application with the device.

2Hardware development

The vibro-thermal tactile unit is composed of two different components: a vibrotactile unit and a thermal tactile unit. We developed these two components
separately and then integrated them by closely placed side by side.
Vibrator unit Vibrator unit was developed by miniaturizing previously developed actuator called ForceReactorTM AF series short-vibration feedback device [13]. Since previous actuator was relatively large for mobile device use, sothat we developed a prototype that has a smaller dimension. The dimensionof current vibrator prototype was 10 × 3 × 24 mm, which was slimmed down
comparing with previous one (7.5 × 5 × 35 mm). The weight is 6 gram, which
is light enough for integrating with developed thermal tactile unit.
Thermal unit We chose a Peltier device (KSMH029F, KELK Ltd.) as thermal
unit for giving tactile feedback. We used four Peltier devices to provide fast
thermal feedback as proposed previously [12]. The size of each Peltier device is
6.0 × 10 × 1.7 mm, which is small enough even we used four Peltier devices intile patterns as shown in Fig. 1. This configuration enables fast temporal changeof temperature, because two of four Peltier devices are used only for coolingand others are used only for warming the skin. The problem of using Peltier
devices is that it requires more electricity when one tries to cool down after it isheated, or vice versa. Four-tile configuration can solve this conventional problem
and previous study showed that the time to perceive temperature change wasimproved 36 % on average [12].
Vibro-thermal module assembly
We assembled the vibrator with the thermal unit into one module. Fig. 1a isthe schematic of the module, and Fig. 1b is the appearance of the developed
module. Three thermistors were equipped in order to measure the temperatureof warming and cooling Peltier devices including housing base, avoiding too muchcooling and warming during thermal feedback. The thermistor between Peltier
devices works as a reference point for capturing the temperature of contacting
skin. The size of assembled module was 16 × 7 × 32 mm and the weight was 10gram with the housing base.

A novel multimodal tactile module

a Peltier device(for warming)

3b
Peltier device(for cooling)
Thermistor

Vibrator

Fig. 1. Schematics (a) and developed (b) vibro-thermal tactile display

3

Hardware evaluation

3.1

Vibratory characteristics

Hardware We evaluated frequency response of the developed module. We measured the displacement of the module with a laser displacement sensor (Keyence,LK-H055) from the top of the module. The module was fixed onto the stage witha sponge, in order to imitate light contact with a finger pad. We observed tworesonant frequencies around 150 and 500 Hz as shown in Fig. 2a.
Psychological characteristics We also evaluated psychological characteristicsof the developed vibrator.
In the experiment, we measured individual difference of vibrotactile sensitivity for different frequencies. We measured equal-loudness contour in this experiment. Participants were asked to adjust the intensity of comparison vibrotactile
stimulus to intensity of standard stimulus with a ten-key control pad. We used
a 400 Hz of sine wave as standard stimulus. Comparison frequency was one often vibrotactile frequencies (40, 80, 120, 160, 200, 240, 280, 320, 360, 400 Hz).
Once participants judged adjusted intensity of comparison stimulus is the sameas standard stimulus, they terminated one trial and moved on next trial. The
order of presented frequency of vibrotactile stimulus was randomized for eachparticipant. Total number of trials were fifty (ten kinds of vibrotactile frequency
× five trials each) and averaged duration for the experiment was about thirty
minutes. Five participants attended this experiment.
Figure 2b shows averaged amplitude of tactile stimuli that gave equal perceptual intensity with 400 Hz vibrotactile stimuli. X-axis is the presented vibrotactile frequency and Y-axis is the intensity of stimuli in dB unit that gave equal
perceptual intensity.
The data showed apparent lower peak at around 150 Hz, which is due tothe resonance of the vibrator. This would be due to the frequency response ofvibroator itself which had a steep resonant peak at 150 Hz as shown in Fig.
2a. This data also implies that the frequency response of mechanical stimulator
could affect perceptual intensity of vibrotactile stimuli.

4M. Nakatani et al.
ab200

20105dB

Displacement [µm]15150

1000-550-1000100200300

400

500

600

700

800

Frequency [Hz]

900

1000

-154080

120160200240

280

320

360400Frequency [Hz]Fig. 2. Frequency response of (a) mechanical vibrator and (b) its equal-loudness contour estimated by human participants.

3.2

Thermal characteristics

We evaluated thermal characteristic of the developed module. We measured thetemperature of warming and cooling Peltier devices using built-in thermistors asindicated in Fig. 1. In addition, we measured the temperature of bottom side ofvibro-thermal module to observe the trend of heat diffusion in developed module.
In the evaluation experiment, we continuously input current to pairs of warming and cooling Peltier devices respectively in order to alternately increase ordecrease their temperature. The duration of applied current was 1.5 seconds, andthe interval between warming and cooling input was 3.5 seconds. In this experiment, the contact side of warming Peltier devices were always warmed up, andnever cooled down by applying current in opposite directions. Similarly, cooling
Peltier devices were always cooled down. Once the temperature of module bottom reached at 30 ◦ C, we stopped current input to the Peltier devices. The room
temperature was 22 ◦ C, and the sampling rate was 2 Hz in this experiment.
Based on the result shown in Fig. 3, we observed expected increase anddecrease of temperature in both warming and cooling Peltier devices, however,
we also observed the temperature increasing trend in all thermistors. This wasdue to the heat diffusion from both warming and cooling Peltier devices. In
the beginning of the experiment, the temperature of bottom side of the module
increased rapidly until the actuation of warming Peltier devices stopped around
100 seconds, and kept increasing its temperature slowly (Fig. 3). The former
temperature increase would be due to the heat diffusion from warming Peltier
devices, and the latter would be from bottom side of cooling Peltier devices.

4Discussion

In this study we developed vibro-thermal tactile display in a small size by assembling small vibrotactile display and four Peltier devices into one module.
The size of the module is small (32 mm in maximum length) and light (10 gramweight) so that this could be useful for mobile or wearable devices.

A novel multimodal tactile module
1.5 sec
Warming
3.5 sec

Temperature (deg C)

45

3.5 sec

5....

Cooling
1.5 sec

Current input

40

35

30
Peltier (warming)
25

Peltier (cooling)
Bottom of the module

200100200

300

400500600

Time (sec)Fig. 3. Temperature profile of termotactile unit in developed module

Based on the evaluation experiment, the performance of vibrotactile presentation was good in relatively lower frequeny bandwidth (120-160 Hz). This may
be good characteristics for vibrotactile display, because human has relatively
higher sensitivity in this frequency range.
The performance of thermaltactile presentation in the developed module
seemed to be equivalent to previous study (Sato et al. [11]). An issue we realized was that we needed to improve the performance of heat exhaustion of themodule. This issue is always with thermal tactile displays, however, we plan toovercome this difficulty by using materials that can easily absorb heat from thehousing base.
Vibro-thermal tactile display would increase the possibility of presenting material property in tactile modality. As discussed in previous literature, temperature information is highly associated with recognizing materials through skinsensations. Combining with vibrotactile presentation, we may increase the variety of tactile materials using our developed module. In our preliminary trial,
we added cooling temperature presentation while watching a cup poured waterwith vibrotactile feedback. Our participants told that it was realistic tactile experience, or even reported that their finger felt as if it were wet. This may beperceptual illusion or physically wet because of dew formation, and this empiricalphenomenon is worth studying in future experiment.
We also realized that we needed to develop a system that can easily edit
vibro-thermal tactile stimuli. Currently we controlled vibrotactile and thermaltactile display from different software, but an integrated system for editing tactile
stimuli may increase the usability of the module that we developed. This is also
an item of the list for future development of this series of study.
Acknowledgments. This research was supported by JST-ACCEL Embodied Media project. Authors thank Dr. Hiroyuki Kajimoto for helpful discussion.

6

M. Nakatani et al.

References
1. Ho, H.N., Jones, L.A.: Contribution of thermal cues to material discrimination and localization. Perception & Psychophysics 68(1), 118–128,
http://dx.doi.org/10.3758/BF03193662
2. Ho, H.N., Jones, L.A.: Development and evaluation of a thermal display for material identification and discrimination. ACM Trans. Appl. Percept. 4(2) (Jul 2007),
http://doi.acm.org/10.1145/1265957.1265962
3. Hribar, V.E., Pawluk, D.T.: A tactile-thermal display for haptic exploration ofvirtual paintings. In: The Proceedings of the 13th International ACM SIGACCESS
Conference on Computers and Accessibility. pp. 221–222. ASSETS ’11, ACM, New
York, NY, USA (2011), http://doi.acm.org/10.1145/2049536.2049577
4. Jimenez, M., Fishel, J.: Evaluation of force, vibration and thermal tactile feedbackin prosthetic limbs. In: Haptics Symposium (HAPTICS), 2014 IEEE. pp. 437–441(Feb 2014)
5. Jones, L., Berris, M.: The psychophysics of temperature perception and thermalinterface design. In: Haptic Interfaces for Virtual Environment and Teleoperator
Systems, 2002. HAPTICS 2002. Proceedings. 10th Symposium on. pp. 137–142(2002)
6. Jones, L.A., Ho, H.: Warm or cool, large or small? the challenge of thermal displays. IEEE T. Haptics 1(1), 53–70 (2008),
http://doi.ieeecomputersociety.org/10.1109/TOH.2008.2
7. Kajimoto, H., Kawakami, N., Tachi, S., Inami, M.: Smarttouch: electric skin totouch the untouchable. IEEE Computer Graphics and Applications 24(1), 36–43(Jan 2004)
8. Killebrew, J.H., Bensmaı̈a, S.J., Dammann, J.F., Denchev, P., Hsiao, S.S., Craig,J.C., Johnson, K.O.: A dense array stimulator to generate arbitrary spatiotemporal tactile stimuli. Journal of neuroscience methods 161(1), 62–74 (3 2007),
http://www.sciencedirect.com/science/article/pii/S0165027006005073
9. Kim, J., Lee, M., Shim, H.J., Ghaffari, R., Cho, H.R., Son, D., Jung, Y.H., Soh, M.,
Choi, C., Jung, S., Chu, K., Jeon, D., Lee, S.T., Kim, J.H., Choi, S.H., Hyeon, T.,
Kim, D.H.: Article stretchable silicon nanoribbon electronics for skin prosthesis.
Nature Communications 5 (2014)
10. Nara, T., Takasaki, M., Maeda, T., Higuchi, T., Ando, S., Tachi, S.: Surface acoustic wave tactile display. IEEE Comput. Graph. Appl. 21(6), 56–63 (Nov 2001),
http://dx.doi.org/10.1109/38.963461
11. Sato, K., Maeno, T.: Haptics: Perception, Devices, Mobility, and Communication:
International Conference, EuroHaptics 2012, Tampere, Finland, June 13-15, 2012.
Proceedings, Part I, chap. Presentation of Sudden Temperature Change Using
Spatially Divided Warm and Cool Stimuli, pp. 457–468. Springer Berlin Heidelberg,Berlin, Heidelberg (2012)
12. Sato, K., Maeno, T.: Presentation of rapid temperature change using spatiallydivided hot and cold stimuli. Journal of Robotics and Mechatronics 25(3), 497–505 (6 2013)
13. Wakuda, H., Kawauchi, T.: Bodily sensed vibration generator system (Jun 3 2004),
https://www.google.ch/patents/US20040104625, uS Patent App. 10/717,070
14. Yamamoto, A., Ishii, T., Higuchi, T.: Electrostatic tactile display for presenting
surface roughness sensation. In: Industrial Technology, 2003 IEEE International
Conference on. vol. 2, pp. 680–684 Vol.2 (Dec 2003)