Twech: A Mobile Platform to Search and Share Visuo-tactile Experiences
Nobuhisa HANAMITSU*1, Haruki NAKAMURA*2, Mhd Yamen Saraiji*1
Kouta MINAMIZAWA*1 and Susumu TACHI*1*1Figure 1.1: Record andGraduate School of Media Design, Keio University*2
Tokyo National College of Technology

Figure 1.2: Example of recorded experience’ data (a scratching cloth) Figure 1.3: Search results

Re-experience device
Overview

Record and Re-experience system

Twech is a mobile platform that enables users to share visuotactile experience and search other experiences for tactile data.
User can record and share visuo-tactile experiences by using avisuo-tactile recording and displaying attachment for smartphone,allows the user to instantly such as tweet, and re-experienceshared data such as visuo-motor coupling. Further, Twech’s
search engine finds similar other experiences, which werescratched material surfaces, communicated with animals or otherexperiences, for uploaded tactile data by using search engine isbased on deep learning that ware expanded for recognizing tactile
materials. Twech provides a sharing and finding haptic
experiences and users re-experience uploaded visual-tactile datafrom cloud server.

A record and re-experience device based on TECHTILE toolkit[Minamizawa et al. 2012], consists of a tactile amplifier, a tactile
microphone that is able to capture frequencies as low as
approximately 10Hz and a tactile actuator. The tactile microphoneequipped bottom of a build-in smartphone camera. Thisequipment helps users to feel experience such as own experience.
To re-experience recorded by users, this system uses visuo-motor
coupling situation and accordingly playbacks the recorded tactile
data. These give us the effect of embodied experience from sharedexperience

User Experience
Users can record haptic experience with in visual image. The
recorded data is visual-and tactile experience by using visuomotor coupling. Users feel just like own experiences by others
shared experiences, for example to scratching material or tocommunicating with animals. When users use search engine, they
can find similar tactile data from an inputted tactile data. Using
this platform, people can capture and share their daily-life
experience of touch to the Social network via smartphone so that
our communication would be more emotional.

Haptic search engine based on Deep Learning
Haptic search finds similar tactile data by using convolutional
neural networks [Krizhevsky et al. 2012] that expanded for tactile
recognition. The input data was used spectrogram and divides
between tactile property (0Hz-1000Hz) and audio property(1000Hz-20kHz). In addition to spatial property can suppose
whether texture is roughness or not from two geometricdimensions of spectrogram. This engine uses those properties,
decides weight values from them, learns deeply and makes theneural network for recognizing tactile data. The learned engine
results similarities that calculated from an uploaded data.

Acknowledgements
This work is supported by JST-ACCEL Embodied Media projectand JSPS KAKENHI Grant #26700018.

Figure 2: Algorithm of tactile recognition*e-mail: contact@tachilab.org
Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor commercial advantage and that copies bear this notice and the full citation on thefirst page. Copyrights for third-party components of this work must be honored. For allother uses, contact the Owner/Author.
Copyright is held by the owner/author(s).
SIGGRAPH 2015 Posters, August 09 – 13, 2015, Los Angeles, CA.
ACM 978-1-4503-3632-1/15/08.
http://dx.doi.org/10.1145/2787626.2792628

References
K.MINAMIZAWA,
Y.KAKEHI,
M.NAKATANI,S.MIHARA,
AND S. TACHI. 2012. TECHTILE toolkit: a prototyping tool for
design and education of haptic media. In Proceedings of the2012 Virtual Reality International Conference, Article 26, 2p.
A. KRIZHEVSKY, I. SUTSKEVER AND G. HINTON. 2012. Imagenet
classification with deep convolutional neural networks,
In Advances in Neural Information Processing Systems, vol. 25,
pp. 1106-1114.