SCIS-ISIS 2012, Kobe, Japan, November 20-24, 2012

Vibrotactile Stimulus Frequency Optimization for
the Haptic BCI Prototype
Hiromu Mori1 , Yoshihiro Matsumito1 , Shoji Makino1 , Victor Kryssanov3 , and Tomasz M. Rutkowski1,2,∗
1

TARA Life Science Center, University of Tsukuba, Tsukuba, Japan
2
RIKEN Brain Science Institute, Wako-shi, Japan3
Ritsumeikan University, Kyoto, Japan∗
Email: tomek@tara.tsukuba.ac.jp
http://about.bci-lab.info

Abstract—The paper presents results from a psychophysicalstudy conducted to optimize vibrotactile stimuli delivered tosubject finger tips in order to evoke the somatosensory responses to be utilized next in a haptic brain computer interface(hBCI) paradigm. We also present the preliminary EEG evoked
responses for the chosen stimulating frequency. The obtained
results confirm our hypothesis that the hBCI paradigm concept
is valid and it will allow for rapid stimuli presentation in orderto improve information-transfer-rate (ITR) of the BCI.

I. I NTRODUCTION
The state of the art brain computer/machine (BCI/BMI)
interfaces relay on visual and imagery paradigms [1], which
require longer training or good vision from the subjects.
Alternative solutions propose to utilize auditory [2] or haptic(somatosensory) modalities [3], [4] in order to enhance braincomputer interfacing comfort or to boost information transfer
rate (ITR).
A concept of utilizing brain somatosensory (haptic) modality creates a very interesting possibility to target tactile sensory
domain, which is not as demanding as vision during operationof machinery or visual computer applications. A potential
haptic/somatosensory BCI/BMI paradigm is thus potentiallya less mentally demanding. The first successful trial to utilize
steady-state somatosensory responses (SSSR) to create the
BCI/BMI [3] targeted a very low stimulus frequency range of
20 − 31Hz to elucidate subject’s attentional modulation. The
lower frequencies excite Meissner-endings of human finger
tips [5]. We propose to utilize also higher frequencies in arange of 300 − 400Hz to stimulate Pacini-endigs [5], sincethese frequencies are very suitable for shorter stimulus delivery
which would fit perfectly a somatosensory evoked potential(SEP) generation mechanism in the brain.
In order to identify user’s preferences before a final
BCI/BMI paradigm design which will utilize stimuli–driven
responses captured in EEG, we conduct first psychophysicalexperiments to identify the preferred stimuli frequencies frombehavioral responses. This paper reports psychophysical results obtained from stimulation of the five fingers and a palm
area of a single hand with various frequencies as depicted in
Figure 1. Next we also present the preliminary results with∗ The corresponding author.

978-1-4673-2743-5/12/$31.00 ©2012 IEEESEP EEG averaged responses to validate our hypothesis of
the hBCI concept.
In the following sections we introduce vibrotactile stimulus
delivery technique. Next we describe the psychophysical andEEG experiments. A statistical analysis of the obtained results
discussion concludes the paper.
II. M ETHODSThe SEP generated by the vibrotactile stimuli itself recordedin brain waves (EEG) is an evoked transient neural potential
caused by the complex tactile stimulus. It is evoked by turning
ON/OFF of a tactile vibration impulse [2], [4].
In order to design comfortable and easy to detect byhuman subjects the vibrotactile stimuli, which next will beutilized in BCI/BMI experiments, we conduct first preliminary
experiments in order to determine stimulus difficulty frombehavioral responses - “button presses” after identified targets.
It is known, that a variability of psychophysical response timedelays is related to task difficulty and the cognitive load.
We aim to determine six vibrotactile stimuli frequencies withuniform task difficulties which could be next plugged into
a haptic BCI/BMI paradigm. Next we also test brain SEP
response in EEG.

Fig. 1. Vibrotactile stimuli delivered to the left hand finger tips and the palm
area using acoustic frequency tactile exciters. The right (dominant) hand ispositioned on a response pad used in the psychophysical experiment.

2150

SCIS-ISIS 2012, Kobe, Japan, November 20-24, 2012

Psychophysical responses of subject #2

700
600

response time [ms]

300

300

400

500

500
400

response time [ms]

600

800

700

900

Psychophysical responses of subject #1

30

50

300

333

366

400

30

50

stimulus frequency [Hz]

333

366

400

stimulus frequency [Hz]Fig. 2. Psychophysical experiment results in form of response time delays
to various vibrotactile stimulus frequencies for subject #1. The differencesamong stimulus frequencies are not significant in the tested frequency ranges.

Fig. 3. Psychophysical experiment results in form of response time delays
to various vibrotactile stimulus frequencies for subject #2. The differencesamong stimulus frequencies are not significant in the tested frequency ranges.

A. Vibrotactile stimuli delivery

Psychophysical responses of subject #3

600
500
300

400

response time [ms]

700

800

The vibrotactile stimuli were delivered as sinusoidal waves
generated by a portable computer with Max/MSP software.
The stimuli were generated via six (for psychophysical experiment) and two (for EEG measurement) channel outputsof an external digital–to–analog signal converter MOTU
UltraLite-mk3 Hybrid coupled with the YAMAHA P4050
power amplifiers. The stimuli were delivered to subject fingers
via two types of tactile exciters working in the ranges of
300 − 20, 000 Hz (four small HIAX19C01-8) and 20 − 80 Hz(two large TT25-16 PUCK) as depicted in Figure 1. The
subjects placed their palms and fingers on the exciters andattended (button–press responded in case of psychophysicalor mentally counted only in case of EEG experiments) only
to the instructed locations, while ignoring the other stimuli.

30

B. The Psychophysical Experiment Protocol
The psychophysical experiments were conducted in order
to investigate stimulus carrier frequency relation on subject
response time and accuracy. The behavioral responses werecollected via a small numeric keypad as depicted in Figure 1.
The subject used their dominant hand for responses, while
the other hand was stimulated vibrotactually with various
frequencies. Each trial was composed of a random order
500 ms stimulation delivered to each finger with inter–stimuli–interval (ISI) of 500 ms. The subject was instructed in eachtrial to attend to one stimuli, while ignoring the others. The
response times were registered with the same Max/MSP patch
which was used for the stimulus generation.
C. The EEG Experiment Protocol
The EEG experiments were conducted in order to confirm
the SEP responses usability for the BCI/BMI paradigm. The

978-1-4673-2743-5/12/$31.00 ©2012 IEEE

300

50

300

333

366

400

stimulus frequency [Hz]Fig. 4. Psychophysical experiment results in form of response time delays
to various vibrotactile stimulus frequencies for subject #3. The differencesamong stimulus frequencies are not significant in the tested frequency ranges.

experimental settings were as follows.
The EEG signals were captured with eight dry electrodes portable wireless system by g.tec (g.SAHARA &g.MOBllab+). The electrodes were attached to following scalp
locations Cz, CPz, POz, Pz, P1, P2, C3, and C4. The recorded
EEG signals were processed by BCI2000 application [6]. The
sampling rate was set to 256Hz, high pass filter at 0.1Hz, thelow pass filter at 40Hz, and notch filter at 50Hz.
The two stimuli types with the lengths of [50, 200] ms,or [150, 200] ms, were delivered randomly to the forefingers

2151

SCIS-ISIS 2012, Kobe, Japan, November 20-24, 2012

TABLE IBCI EEG EXPERIMENT CLASSIFICATION ACCURACY AND
BIT– PER – MINUTE – RATE RESULTS .

Fig. 5. Grand mean average of the psychophysical response results for all
subjects as from the experiments depicted in Figures 2– 4.

of subjects’ both hands. Both of the stimuli types had thesame carrier frequency of 333 Hz. We chose this frequency
since the psychophysical tests (see the summary Figure 5)
suggested that it caused the larger response time variability.
During the experiment, each trial was composed of the single
target and three non–targets stimuli presentation in a random
train repeated ten times. The subject was instructed to attended
to target stimulus while ignoring the other three types. This
procedure was repeated for each stimulus as the target. The offline BCI paradigm experiments were conducted in accordancewith institutional ethical committee guidelines for experimentswith human subjects.
III. R ESULTS AND C ONCLUSIONSThe behavioral results obtained for all subjects separately
are presented in Figures 2-4, in form of boxplots of the
response time distributions. The results analysis have shownthat the vibrotactile stimuli frequency does not influencesignificantly the response delays. We conducted pairwise ttest and Wilcoxon analyses to confirm the observations. In
both cases the means have been confirmed to be similar. The
results rejected a hypothesis of mean differences for various
vibrotactile stimulating frequencies. A grand mean average forall subjects has been also presented in a summary Figure 5.
We conclude the psychophysical experiments with an observation that vibrotaclite carrier frequencies in the rangesof 30 − 50 Hz and 300 − 400 Hz did not influence theinstructed vibrotactile target perception and cognition since
the behavioral responses were the same, or differed nonsignificantly. Only the 333 Hz carrier frequency has caused
larger response variability and it was chosen for the subsequent
tests in EEG experiments.

978-1-4673-2743-5/12/$31.00 ©2012 IEEE

Subject

Number of averages

The best accuracy

BPMR

#1

1

75%

19.81 bit/min

#2

2

50%

2.59 bit/min

#3

1

75%

19.81 bit/min

The EEG experiment results has been presented in Figures 6–8 where evoked responses to targets (top panels); nontargets (middle panels); and distance measures in form ofcorrelation coefficients (bottom panels) indicating the features
separability. For the three subjects we have observed clear
differences in evoked potentials patterns between targets andnon-targets, what is also obvious in correlation coefficient diagrams. Each subject on the other hands had various latenciesof the strongest “aha-response” peaks ranging from around300 − 400 ms. We have presented also in the Figure 9 theresults for a single subject in form of evoked response head
topography and a single channel averaged SEP for targets
and non-targets together with standard deviation error bars,
which identified the statistically significant difference around400 − 500 ms for this particular subject. The signed statisticaldifference r2 graph further confirmed the SEP separability inthe above latency range.
In Table I we also have presented the preliminary
BCI experiment classification results with utilization of the
P300Classifier [7] package with ITR calculated in form ofbit–per–minute–rate (BPMR) as proposed in [8]. The results
for the three subjects are very promising taking into account avery conservative ISI of 600 ms which in our future plannedexperiments will be significantly shortened.
As the final conclusion of the presented experiments wereport on the very promising vibrotactile (haptic) BCI/BMI
modality which is a step forward in creation of the morefriendly paradigms.
ACKNOWLEDGMENTS
This research was supported in part by the Strategic Information and Communications R&D Promotion Programme
no. 121803027 of The Ministry of Internal Affairs and Communication in Japan, and by KAKENHI, the Japan Society
for the Promotion of Science grant no. 12010738. We also
acknowledge the technical support from YAMAHA Sound &IT Development Division in Hamamatsu, Japan.
R EFERENCES[1] J. Wolpaw and E. W. Wolpaw, Eds., Brain-Computer Interfaces: Principles and Practice. Oxford University Press, 2012.
[2] T. M. Rutkowski, “Auditory brain-computer/machine-interface paradigms
design,” in Haptic and Audio Interaction Design, ser. Lecture Notes in
Computer Science, E. Cooper, V. Kryssanov, H. Ogawa, and S. Brewster,Eds. Springer Berlin / Heidelberg, 2011, vol. 6851, pp. 110–119.
[3] G. Muller-Putz, R. Scherer, C. Neuper, and G. Pfurtscheller, “Steadystate somatosensory evoked potentials: suitable brain signals for braincomputer interfaces?” Neural Systems and Rehabilitation Engineering,IEEE Transactions on, vol. 14, no. 1, pp. 30–37, March 2006.

2152

SCIS-ISIS 2012, Kobe, Japan, November 20-24, 2012

Fig. 6. Averaged EEG ERP responses from the subject #1 for targets (the top
panels), non-targets (the middle panels) and in form of correlation coefficients
between the above conditions (the bottom panels). The X − axess represent
time in milliseconds after the stimuli onset and the Y − axes EEG channels.
The figure was created with Py3GUI [9].

Fig. 8. Averaged EEG ERP responses from the subject #3 for targets (the top
panels), non-targets (the middle panels) and in form of correlation coefficients
between the above conditions (the bottom panels). The X − axess represent
time in milliseconds after the stimuli onset and the Y − axes EEG channels.
The figure was created with Py3GUI [9].

464.8438 ms
0.1

0.05

0

EEG [µV]

Channel Cz (targets red; non−targets green)

Fig. 7. Averaged EEG ERP responses from the subject #2 for targets (the top
panels), non-targets (the middle panels) and in form of correlation coefficients
between the above conditions (the bottom panels). The X − axess represent
time in milliseconds after the stimuli onset and the Y − axes EEG channels.
The figure was created with Py3GUI [9].

8
6
4
2
0
−2
0

100

200

300
400
time(ms)

500

0.15

978-1-4673-2743-5/12/$31.00 ©2012 IEEE

0.1
r2[4] T. M. Rutkowski, H. Mori, Y. Matsumoto, Z. Cai, M. Chang,
N. Nishikawa, S. Makino, and K. Mori, “Haptic BCI paradigm based on
somatosensory evoked potential,” Arxiv preprint arXiv:1207.5720, 2012.
[5] R. S. Johansson and J. R. Flanagan, “Coding and use of tactile signals
from the fingertips in object manipulation tasks,” Nature Reviews Neuroscience, vol. 10, no. 5, pp. 345–359, 2009.
[6] G. Schalk and J. Mellinger, A Practical Guide to Brain–Computer
Interfacing with BCI2000. Springer-Verlag London Limited, 2010.
[7] C. M. Potes, “P300Classifier,” 2009. [Online]. Available: http://www.bci2000.org/svn/trunk/src/core/Tools/P300Classifier[8] M. Schreuder, B. Blankertz, and M. Tangermann, “A new auditory
multi-class brain-computer interface paradigm: Spatial hearing as an
informative cue,” PLoS ONE, vol. 5, no. 4, p. e9813, 04 2010. [Online].
Available: http://dx.doi.org/10.1371%2Fjournal.pone.0009813[9] C. R. M. Stocks, “Py3GUI,” 2011. [Online]. Available: https://github.com/collinstocks/Py3GUI

0.05
0

0

200

400

600

time [ms]Fig. 9. The top panel presents a topographic plot of the single subject
SEP response at 464.8438 ms where the largest difference between target
and non-target ERP occurs, as presented in the middle panel together withstander deviation error-bars visualizing the statistical significance (around400 − 500 ms range). The bottom panel presents the signed statisticaldifference (i.e., signed r2 ) value which evaluates the discriminability between
the two types of ERP.

2153