Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.
Digital Object Identifier 10.1109/ACCESS.2017.DOI

GAN-based Fine-tuning of Vibrotactile
Signals to Render Material Surfaces
YUSUKE UJITOKO, 1 , YUKI BAN2 , AND KOICHI HIROTA.3
1
2
3

Research & Development Group, Hitachi, Ltd. and the University of Electro-Communications, Yokohama, Japan (e-mail: yusuke.ujitoko.uz@hitachi.com)
Graduate School of Frontier Sciences, The University of Tokyo, Chiba, Japan. (e-mail: ban@edu.k.u-tokyo.ac.jp)
Graduate School of Information Systems, The University of Electro-Communications, Tokyo, Japan. (e-mail: hirota@vogue.is.uec.ac.jp)

Corresponding author: Yusuke Ujitoko (e-mail: yusuke.ujitoko.uz@hitachi.com).

ABSTRACT The design productivity of fine-tuning for vibrotactile stimuli becomes important as consumer
devices equipped with vibrotactile actuators will become wide-spread. The fine-tuned vibrotactile stimuli
output by vibrotactile actuators allows the end-users to feel the surface of the virtual material. However, there
is no suitable tool for fine-tuning while there are existing tools suitable for initial designing. In this paper,
we test whether we can use GAN (Generative Adversarial Network)-based vibrotactile signal generator
at the tuning phase. The generator provides a material-level interface to designers. Designers can define
any intermediate materials among pre-defined 108 materials and obtain corresponding intermediate signalsthat the generator generates. We showed the applicability of the generator to the fine-tuning of vibrotactile
signals from the viewpoints of principal component analysis and a user test.
INDEX TERMS Human computer interaction, Design tools, Haptic interfaces

I. INTRODUCTIONECENTLY, consumer devices equipped with vibrotactile actuators such as Apple Watch [1] or Pebble
watch [2] have become wide-spread. For example, there aretouch pens, touchpads, and game controllers with vibrotactile actuators around us, and we think of them as nothing
special [3]. Vibrotactile stimulation allows humans to perceive the material properties of virtual surfaces [4]–[7]. If
vibrotactile stimuli match the material attributes of visualized
surfaces in the application, the users perceive the textures
as more realistic [7]. Thus, tactile designers need to design
vibrations carefully and how to design the vibrotactile stimuli
is becoming essential.
The vibrotactile design process consists of preparation,
initial designing, and fine-tuning. It is known that the finetuning phase occupies more than half of all workloads [8].
Thus, improving productivity in the fine-tuning phase willcontribute to the improvement of all workloads. However,there is no suitable tool for iteration and refinement while
there are existing tools for preparation and initial designing [9], [10]. The difficulty in fine-tuning exists in the controlof high-level material parameters such as rocky or metallic by changing low-level engineering parameters such as
frequency or amplitude [11]. The ability to use low-levelengineering parameters to construct or evaluate for material
characteristics is tacit knowledge that vibrotactile designers

R

VOLUME 4, 2016

build over the years.
Based on these considerations, we set a goal of this study
as providing a vibrotactile design method that enables nonexpert designers to fine-tune vibrations with material-leveluser-interfaces. This study utilizes the Generative Adversarial Network (GAN) based vibration generator introduced in aprevious study [12].
At the phase of fine-tuning of the vibrotactile signals using
the generator, we assumed the following scenario. When designers would like to obtain the vibration of intermediate materials of the aluminum board and wooden board, designers
feed interpolated material attributes of the aluminum board
and wooden board into the generator. They get generated
vibrations. However, the generated vibrations do not satisfy
the designers’ demands. In such a case, designers slightly
modify the interpolation ratio of material attributes and they
try a newly generated vibration.
Thus, in this study, we test whether the model allows
designers to synthesize intermediate vibrations by configuring the interpolation ratio of the material attributes. We
evaluated the generated signals from the interpolated material
attributes in both objective and subjective evaluations. The
results of the evaluations showed that the model provides afine-tunability of vibrotactile signals for designers.
II. RELATED WORK
1Ujitoko et al.: GAN-based Fine-tuning of Vibrotactile Signals to Render Material Surfaces

A. EXISTING VIBROTACTILE DESIGNING TOOL

B. GENERATIVE ADVERSARIAL NETWORK

1) Direct Playback of Recorded Vibrotactile Signal

GANs [23] is a popular generative framework that can beused to generate new samples. GANs framework is mostly
used for generating realistic samples of natural images. In
this study, we use the GANs framework to generate sharp
time-frequency samples for vibrotactile feedback. Specifically, generating the vibrotactile samples from materials
attributes is our goal. Such the cross-modal generation using
GANs has been studied recently. Text to image generation
was realized by [24]. In their studies, plausible images forbirds and flowers were generated from their text descriptions.
Cross-modal audio-visual generation was studied in [25].
Though they tried to generate a spectrogram convertible tosound from video frames, they generated the rough spectrogram, which was not convertible to good sound. The reason
for the poor generation was that even the recent GANs
couldn’t generate the image covering a broad frequency
range such as 44100 Hz in the audio data. In contrast, thefrequency range needed for vibrotactile stimuli is smaller
than audio stimuli, and thus GANs could cover the frequency range or vibrotactile signals that is enough for humanmechanoreceptors.

The most simple design method is to record and replay
the scanned acceleration signals [13]–[15] of the material
surfaces. In this method, designers record raw signals withmicrophones or acceleration sensors while they touch thesurface of the material. TECHTILE toolkit and Stereohaptics provided recording devices that can be easily accessedby designers [13], [14]. They held workshops where nonprofessional people were encouraged to work with vibrotactile recording and replaying experience. Seteohaptics [15]adopted the same approach as the TECHTILE toolkit. They
utilized the left and right audio channels and designed two
vibrations at once. Since these studies recorded the actual vibration occurred at the contact surface of the material, replaying the recorded signals can give users high realism throughthe vibration. The disadvantages of this design method arethat it is time-consuming and has little flexibility. Because
the designers need to prepare the material at hand, it takes
much time to record vibrations. In addition, if the recorded
vibrations do not satisfy the requirements a little, the designers need to find similar but different material surfaces to finetune the vibrations.
Recently, open datasets which are a collection of vibrotactile signals have appeared [16]–[19]. Designers can obtain 100 kinds of texture vibrations from The Penn Haptic
Texture Toolkit [16], which is based on the recorded data.
The LMT haptic texture database [17] provides recorded
vibrations of 108 different materials. By utilizing these open
vibrotactile datasets, designers do not need to prepare actual
materials and recording systems themselves. However, thecurrent datasets contain vibrations of up to hundreds ofmaterials, though there exist unlimited kinds of materials in
the real world. Therefore, it is difficult to find the appropriate
vibrotactile signals from the datasets currently.
2) Vibrotactile Authoring Tool

With their crucial role in the vibrotactile design process,authoring tools have received increasing attention in recent
years. Existing tools are built around the most importantdesign parameters and approaches identified in the literatureor by practitioners. Such tools use graphical, mathematical representations to edit the characteristics of the wave,
including the amplitude, frequency, and duration of tactile
sequences [9], [10], [20]–[22]. For example, a web-based
system [21] is an opportunity to edit and observe the effect of high availability. These studies provided a low-level,
simple waveform timeline interface for combining them into
vibrotactile patterns. However, it is difficult to express a highlevel material impression using the combination of simply
patterned waveforms. Therefore, it is not easy for designers
to design vibrations that give users high-realism of material
properties.

2(a) Training

material c
vectorrandom z
vector(b) Inferencematerial c
vectorrandom z
vectordatasetGenerator

true
spectrogram

Discriminatortrue or generated?
inferredmaterial vector cgenerated
spectrogramused for inference

signalGenerator
generated
spectrogram

griffin-Lim
Algorithm

FIGURE 1. Top(a): Generator and discriminator are connected during thetraining phase. Bottom(b): Generator architecture during inference phase.
Only the generator is used for the inference phase.

Thus, we have constructed the vibrotactile signal generator based on GANs. The designers can obtain the corresponding signals by feeding the material vectors into thegenerator. The previous study [12] demonstrated the concept of a training model that can generate vibrotactile signals. The output of the generator is vibration and it isan acceleration signal (shown in Fig.1). The input to thegenerator is the material vector c representing the material attributes of the vibrotactile signal. The material vector indicating the attribute of the material is, for example, (ctimberA , cmetalB , cf abricC , · · · ) = (1.0, 0, 0.0, · · · ),(ctimberA , cmetalB , cf abricC , · · · ) = (0.3, 0, 0.1, · · · ), and soon.
The dimensions of the material vector are 108, and thus designers can designate vector in the 108 material dimensions.
However, whether the model can generate the intermediate
VOLUME 4, 2016

Ujitoko et al.: GAN-based Fine-tuning of Vibrotactile Signals to Render Material Surfaces

signals from multiple materials has not been clarified yet.
Thus, this study evaluates the capability of the model at thefine-tuning phase to provide intermediate signals as follows:• In section III, we clarify whether the model can generatesignals monotonically and continuously with regard to
the transition of the material vector by means of the
Principal Component Analysis (PCA).
• In section IV, we conduct a user study to clarify whether
the user feels the monotonous change in the material
feeling of vibration when the input material vector isfinely and monotonously changed.
III. OBSERVATION OF GENERATED, INTERMEDIATEVIBRATION USING PCA

At the fine-tuning phase, designers slightly change the interpolated material vector and feed them into our generator.
The generator is required to generate the corresponding
continuously changed vibrotactile signals according to thechanged material vector. In this observation, we assumed thesimplest case where designers use the interpolated material
vector cinterpolation between the material vectors c1 and c2
representing two different specific materials.
cinterpolation = c1 · (1 − α) + c2 · α(1)vibration generated for each of 108 materials are shown in
Fig.2. Vibrations generated from material vectors directly
representing 108 different materials are plotted in different
colors. The horizontal axis corresponds to the first principle
component and the vertical axis represents the second principle component. The figure shows the distance between thegenerated vibrations. For example, the generated signal of“Teak” material is different from those of other materials inthese two main components.
In order to qualitatively evaluate the fine-tunability, we
selected some pairs of materials to use their material vectors
as c1 and c2 . We selected “Teak” as a material of c1 and fiveother materials of c2 , the material names of which are shownin Fig.2. The reason for the selection of these materials is
because their positional relations were clear in the PCA. The
generated signals corresponding to six materials are shown in
Fig.2.
We visualize the intermediate spectrograms that are generated based on interpolated material vectors in Fig.3. The
leftmost column shows the generated spectrogram based on
c = c1 = cT eak . The rightmost column shows the spectrogram generated when c = c2 . The middle columns shows
the spectrogram generated when c is intermediate between
c1 and c2 .

For example, when designers would like to get a vibration intermediate between that for materials “Jeans” and“Aluminum”, they synthesized a material vector by mixing
cJeans and cAluminum and fed this into the generator. Then,
designers slightly change the α to fine-tune the intermediate
vibration. We verify the vibration generated in that case.
Teak

GlitterPaper
FineArtificial Version1
GrassFibers

Larch

Core
CeramicPlate Beech

FIGURE 3. Spectrograms generated from intermediate material vectors.

FIGURE 2. Results of the principle analysis.

Then, we use PCA over spectrograms to observe the generated, intermediate vibrotactile signals in comparison withthe vibrotactile signals corresponding to specific materials.
Based on the fact the human vibrotactile sense depends on
specific frequency ranges and on the fact that the PCA overspectrogram shows a variable representative of the frequency
domain, the PCA will provide meaningful insights for theintermediate vibrotactile signals. The PCA results for the
VOLUME 4, 2016

We mapped the generated spectrogram when c was interpolated between cT eak and c2 onto the previous PCA 2dimensional spaces (Fig.4). α is changed from zero to one by
0.05 interval and thus, there are 200 generated spectrogramsfor each pair of cT eak and c2 . The intermediate spectrograms
are colored in blue at the point of “Teak” when α is zeroand c = cT eak . They are colored in red when α is one andc = c2 . The continuous transition from blue to red shows thecontinuous transition of the generated, intermediate signals
as α increases.
Summarizing the observation, with regard to the first andsecond main components of PCA, it is shown that the generated, intermediate signal has an intermediate characteristic
between c1 and c2 . In addition, because the point in blue
3

Ujitoko et al.: GAN-based Fine-tuning of Vibrotactile Signals to Render Material Surfaces

FIGURE 4. Spectrograms generated from intermediate material vectors are
mapped on the PCA two-dimensional spaces.

changes to the point in red continuously, the generated vibrations are shown to be distributed continuously. This showsthat designers can tune the vibration by changing the material
vectors.
IV. USER STUDY

We conducted the user study to test whether our generator
could provide the fine tunability of generated vibrotactile
signals from a perceptual point of view. The number ofparticipants was ten (eight males and two females) and wereaged from 22 to 25 years. All the participants were righthanded. None of them reported a history of neurological,
psychiatric or other diseases that could have interfered withtactile sensitivity. The University of Tokyo Ethics committee(approbation number: KE17-63) approved the data acquisition in this paper and written informed consent was obtained
from all participants.
A. EXPERIMENTAL SYSTEM

The participant task in this user study was to move a pentype device on the surface of a tablet screen while receiving
vibrotactile feedback via the pen. Our experimental system
included a tablet device (Apple Inc., iPad Pro 9.7 inch), a signal amplifier (Lepai Inc., LP-2020A ), and a pen-type device
with a vibrator (Fig.5). The tablet device’s screen refresh rate
was 60 Hz, and the touch position acquisition was performed
at 100 Hz. Although no formal data were obtained on theaccuracy of the contact position acquisition, the resolution
was 264 ppi and the contact position was acquired every 1
pixel; hence, it is assumed that the acquisition accuracy isabout 0.1 mm. The pen-type device is specifically describedin the next paragraph.
The pen device was approximately 140 mm long and theweight of it was approximately 20 g. The diameter of the grip
part of the pen was approximately 10 mm. We covered thepen tip with a conductive material that is ordinarily used forthe stylus. Because the shaft of the pen was plastic and does
not conduct to the grip part, we wound a conductive sheeton the grip to react with a capacitance-type touch screen.
We embedded the vibrator (ALPS Inc., HAPTIC Reactor
4

FIGURE 5. Setup of the experimental system.

Hybrid Tough Type [26]) inside the pen-type device at a
position 2 cm from the end of the pen where the participants
gripped it. The vibrator was small (35.0×5.0×7.5 mm) andlight (approximately 5 g) enough that participants would nottire from moving the pen.
When participants moved the pen on the surface, the vibration signal was emitted from the earphone jack of the tablet.
The amplifier amplified the signal and the vibrator embeddedin the pen presented the vibration to the participants’ fingers.
B. VIBRATION PRESENTED TO PARTICIPANTS

To apply the evaluation result of the user study to materials as
diverse as possible, we selected four pairs of base materials
based on four different points of view. We assumed that thebase materials were either 1) the same material, 2) similarmaterials, 3) not similar materials but intermediate material
is familiar to users, or 4) totally different materials for whichintermediate material does not exist. The four viewpoints
have differences in the similarity between materials or the
probability of presence in the real world of the intermediatematerials between the two. We used pairs of material vectors
c1 and c2 of the base materials and synthesized the intermediate material vector c using equation (1). The four pairs ofbase materials selected were as follows:1) c1 : “Squared Aluminum Mesh”, c2 : “Aluminum Plate”(Same materials but different local geometry)2) c1 : “Carpets”, c2 : “Jeans”(Similar materials)3) c1 : “Aluminum Plate”, c2 : “Granite Type Veneziano”(Not Similar materials)4) c1 : “Jeans”, c2 : “Granite Type Veneziano”(Totally different materials for which intermediate material does not exist)
The difference between 3) and 4) is the probability ofthe presence of intermediate materials. The intermediatematerials between “Aluminum Plate” and “Granite Type
Veneziano” can be present as an alloy or ore mineral. In
contrast, the mixture between “Jeans” and “Granite Type
Veneziano” never be present.
We synthesized variations of the material vector c by
VOLUME 4, 2016

Ujitoko et al.: GAN-based Fine-tuning of Vibrotactile Signals to Render Material Surfaces

setting α at 0.0, 0.2, 0.4, 0.5, 0.6, 0.8, and 1.0. When α wasclose to zero, the material vector c was close to c1 . When α
was close to one, the material vector c was close to c2 .

total. To prevent sequential effects, the presentation orderof the intermediate vibrations was randomly assigned andcounterbalanced across participants. Also, the assignment of
c1 and c2 for the first and second paths is randomized.

C. TASK DESIGN
D. RESULTS

Participants
move onthree paths

Answer
ButtonYardstick ofa movement speed

FIGURE 6. Experimental window.

This study used a within-participants design. Participants
could move the pen-type device along the three different
predefined paths displayed on the screen. The experimental
window is shown in Fig.6. On the first path, participants felt
vibrations generated from the material vector c1 or c2 . On thesecond path, they felt vibrations generated from the material
vector c1 or c2 . When vibrations from c1 were set on thefirst path, vibrations from c2 were set on the second path.
When the vibrations from c1 were set on the second path, thevibrations from c2 were set on the first path. On the third path,they felt vibrations generated from the interpolated material
vector c. The task assigned to the participant was to judge
whether the intermediate generated vibration on the third
path from the material vector c was similar to the vibrations
on the first or second path.
In the following paragraph, we describe the procedure of
the participants’ task in one trial. The participants moved
the pen on a predefined path from left to right for about
100 mm distance at a fixed speed with their dominant hand.
The touchscreen visualized a bar that indicated at what speed
to move the pen in order to control the movement speed.
Participants moved the pen over a distance of approximately
100 mm in 1.6 seconds, which is what the bar elongation
instructed the participants to do. Participants were asked togrip the pen with their fingers at the position where the vibrator was embedded so that they could feel the vibration with
their fingers. They answered which generated vibration (i.e.,on the first or second path) was similar to the intermediate
generated vibration on the third path. They tapped one of thetwo answer buttons visualized at the bottom of the screen.
For each of four pairs, participants judged the seven intermediate vibrations (α is 0.0, 0.2, 0.4, 0.5, 0.6, 0.8, and1.0) and they judged each intermediate vibration seven times.
Thus, each participant did 196 (= 4 × 7 × 7) trials in
VOLUME 4, 2016

FIGURE 7. Percentage of answers that said the intermediate vibration wasclose to the vibration generated from the material vector c1 . The error bar
represents the standard error computed from the average percentage ofparticipants.

Fig.7 shows the percentage of participants’ answers sayingthat the intermediate vibration was close to the vibration generated from the material vector c1 . The percentage changed
monotonically according to the configuration of α for all
pairs.
One of the free comments from participants said that the
texture felt much different from the glass surfaces of thetouchscreen.
E. DISCUSSION

For all pairs (1–4), the percentage of answers seems totransition monotonically. This means that the generated intermediate vibrations from two different materials can becontrolled by the α value. Let us assume the cases when thetactile designers generate the intermediate vibration from acertain base materials "A" and "B" by defining the α value.
After setting the α at 0.5, if designers try the generated
vibration and would like to obtain a generated vibration close
to material "A", they should set α to a smaller value such as
0.4. When they would like to obtain the generated vibration
close to material "B", they should set α to a larger value suchas 0.6.
Although the monotonicity of the curve in Fig.7 is thesame, the points of subjective equality between c1 and c2
were biased for the pairs of 2) “Carpets” and “Jeans” and 4)“Jeans” and “Granite Type Veneziano”. In other words, theα values corresponding to the rate of 0.5 are different among
pairs.
The reason for the bias for the pair of 2) “Carpet” and“Jeans” is thought to be that it was difficult for participants to5

Ujitoko et al.: GAN-based Fine-tuning of Vibrotactile Signals to Render Material Surfaces

imagine the surface vibration of “Carpet” rather than “Jeans”.
Therefore, it seems that participants tended to choose "Jeans"
when α was 0.5.
The reason for the bias for the pair of 4) “Jeans” and“Granite Type Veneziano” is thought to be the difference inthe vibration characteristics for each material. The “Granite
Type Veneziano” was rocky type material and it had specificcharacteristics in lower frequency ranges. This seemed tomake it more frequent for the participants to select “Granite
Type Veneziano” when they felt the vibration in the lowfrequency range. In contrast, the “Jeans” had little characteristics on specific frequency ranges and it seemed to makeit less frequent for the participants to select “Jeans” than
“Granite Type Veneziano”. This leads to the bias of the pointof subjective equality for the pair.
These suggest that the value of α does not uniquely determine the subjective ratings on the intermediate vibrations
among multiple materials. Thus, the process of tuning of
α repeatedly is still needed. However, the low-level designof parameters such as frequency or amplitude is no longer
necessary with our generator. The designers do not need tocare for those low-level design but only care for the materiallevel attribute design. This leads to improved productivity foreach iterative process.
V. CONCLUSIONSThis study evaluates the capability of the GAN-based model
to provide intermediate signals and it can be used during finetuning. The model hides the low-level engineering parameters such as frequency and provides designers material-level
user interface. Thus, non-expert designers can use the model.
The summarization of the evaluation is as follows:• We clarified that the model can generate signals monotonically and continuously with regard to the transitionof material vector by means of the PCA.
• The user study clarified that the user feels themonotonous change in the material feeling of generated
vibration when the input material vector is finely andmonotonously changed.

[9] C. Swindells, S. Pietarinen, and A. Viitanen, “Medium fidelity rapid
prototyping of vibrotactile haptic, audio and video effects,” In Proceedingsof IEEE Haptics Symposium, pp. 515–521, 2014.
[10] J. Ryu and S. Choi, “posvibeditor: Graphical authoring tool of vibrotactile
patterns,” In Proceedings of IEEE International Workshop on Haptic
Audio visual Environments and Games, pp. 120–125, 2008.
[11] H. Seifi, Deriving Semantics and Interlinkages of Facets. Cham: Springer
International Publishing, 2019, pp. 71–109.
[12] Y. Ujitoko and Y. Ban, “Vibrotactile signal generation from texture images
or attributes using generative adversarial network,” in Proceedings of theEuroHaptics 2018.
[13] K. Minamizawa, Y. Kakehi, M. Nakatani, S. Mihara, and S. Tachi,“TECHTILE toolkit,” in Proceedings of the Virtual Reality International
Conference, 2012, p. 1.
[14] M. Nakatani, Y. Kakehi, K. Minamizawa, S. Mihara, and S. Tachi, Techtile
workshop for creating haptic content. Springer Japan, 1 2016, pp. 185–200.
[15] A. Israr, S. Zhao, K. McIntosh, Z. Schwemler, A. Fritz, J. Mars, J. Bedford,
C. Frisson, I. Huerta, M. Kosek, B. Koniaris, and K. Mitchell, “Stereohaptics: A haptic interaction toolkit for tangible virtual experiences,” Inproceedings of ACM SIGGRAPH 2016 Studio, pp. 13:1–13:57, 2016.
[16] H. Culbertson, J. Unwin, and K. J. Kuchenbecker, “Modeling and rendering realistic textures from unconstrained tool-surface interactions,” IEEE
Transactions on Haptics, vol. 7, no. 3, pp. 381–393, 2014.
[17] M. Strese, C. Schuwerk, A. Iepure, and E. Steinbach, “Multimodal
Feature-Based Surface Material Classification,” IEEE Transactions on
Haptics, vol. 10, no. 2, pp. 226–239, 2017.
[18] “Drv2605 haptic driver for erm and lra with built-in library and smart-loop
architecture,” http://www.ti.com/lit/ds/symlink/drv2605.pdf.
[19] H. Seifi, K. Zhang, and K. E. MacLean, “Vibviz: Organizing, visualizing
and navigating vibration libraries,” In Proceedings of IEEE World Haptics
Conference, pp. 254–259, 2015.
[20] C. Swindells, S. Pietarinen, and A. Viitanen, “Medium fidelity rapid
prototyping of vibrotactile haptic, audio and video effects,” In Proceedingsof IEEE Haptics Symposium, pp. 515–521, Feb 2014.
[21] O. S. Schneider and K. E. MacLean, “Studying design process andexample use with macaron, a web-based vibrotactile effect editor,” In
Proceedings of IEEE Haptics Symposium, pp. 52–58, 2016.
[22] D. J. Meyer, M. A. Peshkin, and J. E. Colgate, “Tactile paintbrush: A
procedural method for generating spatial haptic texture,” In Proceedingsof IEEE Haptics Symposium, pp. 259–264, 2016.
[23] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley,
S. Ozair, A. Courville, and Y. Bengio, “Generative Adversarial Nets,” In
Proceedings of Advances in Neural Information Processing Systems, pp.
2672–2680, 2014.
[24] S. Reed, Z. Akata, X. Yan, L. Logeswaran, B. Schiele, and H. Lee,
“Generative Adversarial Text to Image Synthesis,” In Proceedings of
ICML, pp. 1060–1069, 2016.
[25] L. Chen, S. Srivastava, Z. Duan, and C. Xu, “Deep Cross-Modal AudioVisual Generation,” CoRR, 2017.
[26] “Haptic reactor,” https://www.alps.com/prod/info/E/HTML/Haptic/.

REFERENCES[1] “Apple watch,” http://www.apple.com.
[2] “Pebble watch,” http://www.pebble.com.
[3] H. Culbertson, S. B. Schorr, and A. M. Okamura, “Haptics: The present
and future of artificial touch sensation,” Annual Review of Control,
Robotics, and Autonomous Systems, vol. 1, no. 1, pp. 385–409, 2018.
[4] M. Granieri and A. Estrada, “Down converter characterization in a synthetic instrument context,” In Proceedings of IEEE Autotestcon, pp. 20–
25, 2006.
[5] S. Okamoto, H. Nagano, and Y. Yamada, “Psychophysical dimensions oftactile perception of textures,” IEEE Transactions on Haptics, vol. 6, no. 1,
pp. 81–93, 2013.
[6] M. Hollins, A. Fox, and C. Bishop, “Imposed vibration influences perceived tactile smoothness,” Perception, vol. 29, no. 12, pp. 1455–1465,
2000.
[7] H. Culbertson and K. J. Kuchenbecker, “Importance of matching physicalfriction, hardness, and texture in creating realistic haptic virtual surfaces,”
IEEE Transactions on Haptics, vol. 10, no. 1, pp. 63–74, 2017.
[8] O. S. Schneider and K. E. MacLean, “Studying design process andexample use with macaron, a web-based vibrotactile effect editor,” In
Proceedings of IEEE Haptics Symposium, pp. 52–58, 2016.
6

YUSUKE UJITOKO is a researcher at Hitachi,Ltd., Japan and a doctor course student at the
University of Electro-Communications. He received a B.E. degree in mechanical engineeringand M.A.E. degree in inter-disciplinary information studies from the University of Tokyo, Japan,
in 2014 and 2016. His research interests include
applied haptic perception and haptic interfaces.

VOLUME 4, 2016

Ujitoko et al.: GAN-based Fine-tuning of Vibrotactile Signals to Render Material Surfaces

YUKI BAN received the M.S. degree and the
Ph.D. degree in Information Science and Technology from the University of Tokyo, Tokyo,
Japan, in 2013 and 2016, respectively. He wasa researcher at Xcoo inc. research from 2016 to2017. He is currently an assistant professor of
Department of Frontier Sciences at the University of Tokyo. His current research interests focus
on modifying spatial perception using the visuohaptic interaction. He is a member of the IEEE.

KOICHI HIROTA received the B.S. degree fromthe University of Tokyo, Japan, in 1988. He received his Ph.D. degree from the University ofTokyo, in 1994. He was then an assistant professor
with the Toyohashi University of Technology in
1995. In 2000, he was an associate professor with
the University of Tokyo. Currently, he is a professor in the Department of Human Media Systems,
Graduate School of Information Systems, University of Electro-Communications. His researchinterests include haptic rendering and human interfaces. He received several
academic awards.

VOLUME 4, 20167