P300 Responses Classification Improvement in
Tactile BCI with Touch–sense Glove
Hiroki Yajima∗ , Shoji Makino∗ , and Tomasz M. Rutkowski∗,†,5∗ Department of Computer Science and Life Science Center of TARA, University of Tsukuba, Japan

E-mail: tomek@bci-lab.info (5 corresponding author)† RIKEN Brain Science Institute, Japan

Abstract—This paper reports on a project aiming to confirm
whether a tactile stimulator “touch–sense glove” is effective for anovel brain–computer interface (BCI) paradigm and whether thetactile stimulus delivered to the fingers could be utilized to evoke
event related potential (ERP) responses with possible attentional
modulation. The tactile ERPs are expected to improve the BCI
accuracy. The proposed new stimulator device is presented indetail together with psychophysical and EEG BCI experiment
protocols. Results supporting the proposed “touch–sense glove”
device are presented in form of online BCI classification accuracy results. Finally, we outline the future possible paradigm
improvements.

I. I NTRODUCTION
Many researchers recently join various projects related toa brain computer interface (BCI) technology which is expected to allow operation of any device using brainwaves
only [1]. This technology shall allow disable people, e.g. theamyotrophic lateral sclerosis (ALS) users, to operate devices
without any muscle activity necessary. The most popular
BCI is a visual one in which user’s control commands are
estimated only from presented intentional responses to visual
stimulus [1], [2], [3]. This modality, however, prevents users
from paying attention to surrounding environment causing
often difficulties in an application operation. Such BCI is notavailable also for users suffering from lost or bad vision [4].
Our research project proposes to use tactile BCI (tBCI) modality. This modality shall derive P300 response which is usuallyobtained by attending to a specific and known target [1].
Although the auditory modality [5], [6], which is also analternative to the vision, could also derive the P300 responses,
it could not be used in case of advanced ALS patients (e.g.
totally–locked–in syndrome) [6], [4]. Our research aims to
improve tBCI classification accuracy and to develop a practicalstimulation device. We search for the most suitable patterns
leading to a successful multi command tactile paradigm.
The tBCI paradigm often utilizes a somatosensory steady–state response (SSSR) [7]. While a person is stimulated, thebrain generates a response which has nearly the same frequency as the stimulus. The SSSR response becomes stronger
when a person attends to the stimulus [7]. However, the SSSR
does not appear when the stimulation period is very short,thus it is very difficult to gain a good information transfer
rate (ITR) [1]. Recently BCI which utilizes an event related
potential (ERP) instead of the SSSR has has been actively
researched [6], [2], [8], [3], [4]. The ERP features are also

978-616-361-823-8 © 2014 APSIPA(a) Touch-sense glove(b) Exciter positions

Fig. 1. Panel (a) presents the touch–sense glove on a used hand with 12
vibrotactile exciters attached along the fingers. Panel (b) presents detailed
locations on a hand of the 12 vibrotactile exciters.

very suitable to classify a so called “aha–response” (known
also as a P300 response since it is a positive ERP deflection
after about 300 ms from the stimulus onset [1]). So far,the most popular tBCIs use fingertip stimulation to evoke
P300 responses [9]. However, this modality’s accuracy is still
too low for practical utilization, thus, we propose to expand
conventional stimulation to the whole finger surfaces.
The rest of the paper is organized as follows. In the next
section we introduce methods used and developed within
the project. Next, the obtained results and a discussion arepresented. Conclusions and a future research direction outline
summarize the paper.
II. M ETHODSIn this section, we explain details of conventional and our
proposed tBCI paradigms. In our novel tactile BCI paradigm
project we conducted psychophysical and EEG experiments
in order to compare new results with conventional methods.
The psychophysical and EEG experiments were conductedin agreement with the ethical committee guidelines of theFaculty of Engineering, Information and Systems at University of Tsukuba, Tsukuba, Japan (experimental permission
no. 2013R7). Five volunteer users participated in the experiments. We propose to utilize P300 response in the tBCI
paradigm and a classification method based on a classicalAPSIPA 2014

Fig. 2. ARDUINO DUE micro–controller board with output ports connected toour own design 12−channels amplifier. The vibrotactile exciters are connectedto the amplifier outputs and powered for security from a black battery packin the upper part of the photograph. The communication with our experimentmanaging MAX 6 visual programming environment patch is established via
an USB port with RS232 serial communication embedded protocol.

oddball paradigm [1]. The P300 response is a positive deflection starting at around 300 ms after the user attends
stimulus and it does not appear to the ignored one [1]. Usually
the P300 responses are applied to visual and auditory BCI
modalities [2]. The P300–based BCI discriminates the attended
and ignored stimuli from the differences in ERP shapes. An
example of the averaged P300 response evoked to an expected
target stimuli in comparison to the ignored non–targets ispresented in Figure 5. In our experiments, we assign label
to the attended stimulus as the target and to the other ignored
as the non–target.
We proposed a new stimulator made of a glove which
we named “a touch–sense glove” as shown in Figure 1(a).
The glove had embedded 12 vibrotactile exciters which wereattached to user finger are as depicted also in Figure 1(a).
A detailed placement of the exciter positions is outlined in
Figure 1. The reason why the vibrotactile excites were attached
to a glove was to enhance convenience of an experimental
setup avoiding manual attachment of the 12 devices separately
each time. A rubber glove served also as a safety electric
insulator to avoid any current leakage causing a possible
electric interference with EEG.
We first conducted psychophysical experiments in order todetermine the task difficulties based on the recorded behavioral“button press” responses which were executed after the identified stimulus patterns. After that, in order to evaluate the P300(a) Front side attached tofingers.

Fig. 4. The psychophysical experiment instruction screen. The blue rectangle
is a target stimulus pattern. A user responses are collected using a computer
keyboard button–press answers. Correct answer rates are displayed after eachtrial to update the user with accuracy results.

response occurrences and a possible online BCI application
based on online classification, we conducted EEG experimentswith the same users as in the previous psychophysical experiment.
A. Experimental Device Details
The ARDUINO DUE micro–controller board was used togenerate square wave signals delivered to vibrotactile exciters
as presented in Figure 2. The control of the ARDUINO DUE
board was based on a simple program communicating witha portable computer via an USB port with the RS232 serial
communication protocol embedded. The serial communication
with ARDUINO DUE board was managed by a MAX 6
program developed by our team. The vibrotactile exciters (seeFigure 3) used in the experiments were attached to ARDUINO
DUE via a custom made multichannel amplifier developed in
our laboratory as shown in Figure 2. In psychophysical andEEG experiments, there were five stimulus patterns as shownin Table I. The stimulus patterns in form of vibrotactile exciter(b) Back side.

Fig. 3. Tactile exciter HIHX09C005-8 by HiWave used in psychophysical andEEG experiments.

Fig. 5. This figure presents the P300 response which is the positive deflection
after 300 ms from the stimulus onset. The purple line represents the averaged
response to the target stimuli, while the blue to the non–targets.

TABLE III
T HE EEG E XPERIMENT C ONDITIONS
Condition
Number of users
Mean age

100 ms
500 ms5 (thumb; forefinger; middle finger;
ring finger; little finger)

EEG recording systemg.USBamp amplifier and active wet
EEG electrodes system

Sampling ratehigh-pass; low-pass; notch
Reference electrode

TABLE I

Ground electrode

F INGER STIMULUS PATTERN DETAILS USING VIBROTACTILE EXCITERS

Number of sessions
Number of trials in each run

2

3

4

5

Stimulus type

thumb

forefinger

middlefingerring
fingerlittlefinger

1

3

3

3

2

Number of exciters

sets vibrated for 100 ms. Each stimulus instruction pattern was
represented on a user interface display by a blue rectangle asshown in Figure 4.
B. Psychophysical Experiment Protocol
Before the EEG experiments, we conducted the psychophysical study to check whether the users can distinguish tactile apparent motion stimulus patterns. In the psychophysicalexperiments, user button press responses were recorded. Basedon the recorded response datasets, we analyzed the correct
answer rates and the response times. All the psychophysicalexperiment had the same protocol. The experimental procedure
consisted of the following steps:
TABLE II
T HE PSYCHOPHYSICAL EXPERIMENT CONDITIONS
Condition
Number of users
Mean age

Detail5
26.6

Stimulus duration

100 msStimulus frequency

300 HzInter–stimulus–interval (ISI)

500 msStimulus device
Number of sessions for each user
Number of trials in each run

12 vibrotactile exciters2
5 (50 targets
and 200 non–targets)

12 HIHX09C005-8 exciters

Number of stimuli

EEG electrode positions

1

26.6

Inter-stimulus-interval (ISI)

Number of the EEG channels

Stimulus number

5

Stimulus duration
Stimulus device

Fig. 6. A flow diagram of EEG signal processing and brainwaves classifier
training procedures.

Detail8
Cz, CPz, P3, P4, C3, C4,
CP5, and CP6512
0.1 Hz; 60 Hz; 48 ∼ 52 Hz
Behind the user’s left ear
On a forehead (FPz)3
5 (50 targets and 200 non–targets)

1) the visual instruction on a computer display and astimulus to the user finger were given of which pattern
to attend next;2) a sequence of tactile stimuli was delivered to the fingers(an oddball sequence of random ordered patterns);3) the user paid attention only to the instructed pattern (thetarget) while ignoring the others;4) the above three steps were repeated until all the stimulus
patterns become the targets.
The above four steps defined a single sequence. We conducted
the five trials for psychophysical experiments because of thefive different target stimulus patterns used. We assigned all
sequences to a single session and we conducted two experimental sessions for each user. A single trial was composed of
50 randomized order stimuli which consisted the ten targets
and forty non–targets. If the number of button–presses toeach trial was less than the designed number, we did treat
it as a no response of an overlooking condition. Each trial
consisted of the randomized order presentations with fixedinter–stimulus–interval (ISI) and the stimulus durations. All
psychophysical experiments were conducted by a personal
computer running MAX 6 program with the same generated
outputs. The same MAX 6 program registered the behavioral
button–press response times and stimulus numbers to which
the user responded. The stimulus patterns have been summarized in Table I and detailed summary of the psychophysicalexperiment conditions is presented in Table II. The user was
instructed to attend to the target pattern presented in advance
before each random stimulus sequences. The user instructionof which pattern to attend was delivered on a computer screen
as presented in Figure 5. The instruction with the targetdisplayed was changing after each trial. At each trial, the user
could confirm the answer rate success of the executed button–presses. The user generated the behavioral responses with afree second hand.

C. EEG Experiment Protocol
In order to evaluate the P300 response occurrences and theonline BCI classification accuracies, we conducted a series of
EEG experiments with the same users as in the previously
described psychophysical experiment section. The EEG experiments did not require the users to respond behaviorally bypressing a button, but only mental responses were instructed.
Figure 6 presents a flow diagram of EEG signal processing
and brainwaves classifier training pipeline. First, the user’s
brainwaves were captured using wet active EEG electrodes.
Second, the captured and filtered brain signals were segmented
and classified after a training of the step–wise linear discriminant analysis (SWLDA) classifier [10]. Next, the parametersof the trained SWLDA classifier were entered to the BCI2000
software [11] for the subsequent online BCI sessions. EachBCI command output was a result of features drawn fromthe averaged responses and classified by the SWLDA method
within the BCI2000 environment.
The EEG experimental procedure consisted of the the same
steps as in the previous psychophysical experiments. A single
EEG experiment BCI trial was composed of randomized 10targets and 40 non–target stimuli. The EEG signals were
captured with g.USBamp EEG amplifier by g.tec Medical
Engineering, Austria, with 8 wet active electrodes g.LADYbird
by the same manufacturer. The electrodes were attached to thefollowing scalp locations Cz, CPz, P3, P4, C3, C4, CP5, andCP6. The ground was attached to the Fpz and a referenceelectrode on a left earlobe. The recorded EEG signals were
processed by BCI2000 application [11], using the SWLDA
classifier [10]. The sampling rate was set to 512 Hz. A notch
filter to remove power line interferences was set at a rejection
band of 48 ∼ 52 Hz. Next, the EEG signals were digitally
bandpass processed be high–pass and low–pass filters set at
0.1 Hz and 40.0 Hz respectively. A procedure of 10 single
ERP responses averaging was used in order to enhance theP300 amplitudes. In EEG experiment, we also presented thesame instruction screen as shown in Figure 4. P300 (the user
intentions) classification results were presented in form of
numeric values using the same instruction screen. The stimulus
patterns and vibration times have been summarized in Table II.
The EEG recording detailed conditions have been summarizedin Table III. In the EEG experiment, we conducted three runs
for each user. The single run was composed of 50 targets
and 200 non–targets. A single sequence was composed of 10
trials (10 targets and 40 non–targets presented randomly inthe oddball paradigm). The user was instructed to attend to
the target patterns presented in advance before each oddballsequence.

Fig. 7. The psychophysical experiment results in form of a confusion matrix ofthe grand mean averaged user accuracy results. The horizontal axis represents
user response numbers and the vertical the instructed targets respectively. A“no response” column indicates the missed responses by the users. A diagonal
indicates the correct responses, while off diagonals the mistakes. The obtained
accuracy values are also presented with color coding.

Fig. 8. The psychophysical experiment results in form of boxplots of the
grand mean averaged user response times (behavioral button–presses). The
horizontal axis represents the stimulus numbers and the vertical one theresponse times, respectively. No significant differences among the stimulus
patterns were observed.

III. R ESULTS
In this section, we present results of the two experimental
sessions conducted, namely the psychophysical and online BCI
EEG studies outcomes.

Fig. 9. Grand mean ERP averaged results up to 800 ms from the first EEG experiment. Each channel panel presents the averaged ERP plot of the TARGET and
non-TARGET. The purple lines depict TARGETs and blue non-TARGETs respectively. The P300 peaks seem to be in the latency the range of 200 ∼ 600 ms.

A. Psychophysical Experiment Results
As a result of the conducted psychophysical experiments
we obtained user response accuracies and reaction times. A
confusion matrix depicted in Figure 7 was generated based on
the averaged response accuracies of the all five users who
took part in the psychophysical experiments. A horizontal
axis in in Figure 7 represents the stimulus numbers, whilethe vertical one the button-press (behavioral) response times.
A “no response” column has been included to represent theomitted responses. A diagonal line of the confusion matrix
represents correct responses. A color coding has been usedadditional to visualize graphically psychophysical experimentaccuracies. The resulted accuracies were above 90% level,
which was evaluated as a very good outcome and way abovea chance level of 20%. As the result of the psychophysicalexperiment we confirmed the users in our experiments could
distinguish all five vibrotacltile stimulus patterns delivered
using the proposed touch–sense glove. Figure 8 reports response time distributions in form of boxplots. The horizontal
axis in this figure represent the stimulus numbers, while thevertical one the button–press (behavioral) response times inmilliseconds. This boxplot shows that the users reacted totarget stimuli with median time of about 300 ms. No significant
differences were observed among the response times to various
patterns as tested with ANOVA.

TABLE IV
T HE EEG EXPERIMENT BCI CLASSIFICATION ( THEORETICAL CHANCELEVEL OF 20%) AND ITR SCORES ( THE AVERAGED BEST ITR WAS 3.94)

user number

The best classification
accuracy

Averagedaccuracy

The best ITR

1

100%

100%

4.64

2

100%

90%

4.64

3

100%

60%

4.64

4

100%

70%

4.64

5

60%

40%

1.10

B. EEG Experiment Results
In this section, we present the online BCI EEG experimentresults. The brainwaves have been depicted in form of grandmean averaged (all users and sessions) ERPs in Figure 9
and as the area under the curve (AUC) of the EEG features
discrimination analysis results with head topographic plots in
Figure 10. The above results clearly indicated P300 response
validity and potential separability as indicated by AUC values
above 0.5 benchmarks in Figure 10 in the latencies of 250 ∼550 ms. Online tBCI experiments results are also summarized
in form user achieved accuracies (a chance level was of20%) and information transfer rates, which were calculated

Fig. 10. Grand mean AUC averaged results from the first EEG experiment. The top head topographic plots present the AUC scores spatial distributions and the
maximum (here 365 ms) and minimum (here 184 ms) latencies. The second and third from the top matrix plots depict the TARGET and non-TARGET grandmean averaged responses for all EEG channels used in the experiment. The bottom panel presents the AUC scores matrix for TARGET versus non-TARGET
distributions separability evaluation.

as follwos
IT R = V · R,(1)

ITR results would allow for a slow yet already comfortableinteraction using the proposed tBCI paradigm.

where V was the classification speed in selections/min and R
the number of bits/selection calculated as,

IV. C ONCLUSIONS

1−P
,
(2)
N −1
with N was a number of classes (five in this study) and P theobtained in online BCI experiments classification accuracy.
The accuracy and ITR results are listed in the Table IV. The
averaged values reported in Table IV are based on arithmetic
means of the brainwave classification accuracies. The bestclassification accuracy was the maximum accuracy the user
could score in the whole experiment. The averaged accuracy
was the mean from the two sessions each user conducted. We
also presented the maximum accuracies with scores reaching
100% for four out of five users taking part in the study. The
lowest averaged accuracy was of 40% which was still above
the chance level of 20% in the presented study. The obtained

In this paper, the psychophysical and EEG experiments wereconducted in order to confirm our research hypothesis of thenovel stimulator named “the touch–sense glove” usability forthe tBCI. In the series of psychophysical and EEG experiments we confirmed that the users could distinguish the five
vibrotactile stimulus patterns delivered to the five fingers ofthe dominant hand. We could also observe clear and possible
to discriminate brainwave P300 responses.
The tBCI concept was evaluated in online classification
experiments with ten trails averaging setup using SWLDA
classifier of the P300 responses resulting with the final commands.
The obtained results have shown that the averaged classification accuracies resulted above the chance level of 20%.

R = log2 N + P log2 P + (1 − P ) log2

The online tBCI averaged accuracy results were in a range of
40% ∼ 100%. The best obtained ITR was of 4.64 bit/min.
As a result of the conducted study, we could draw thefollowing conclusions:1) The results of the conducted psychophysical experiments
showed that the users could distinguish the tactile stimulation generated by the proposed “touch–sense glove.”2) We could confirm the clear P300 responses in EEG
experiments in which tactile stimulation was generated
by the “touch–sense glove.”3) The EEG experiments resulted with the easily discriminable P300 responses leading to the classification
accuracies and ITR scores above the chance levels, oreven with perfect scores within the limitations of theexperiential settings.
We plan to continue this line of research in the near future
to conduct experiments with shorter ISIs and with single trial–
based classification sequences.
ACKNOWLEDGEMENTS
This research was supported in part by the Strategic Information and Communications R&D Promotion Program
no. 121803027 of The Ministry of Internal Affairs and Communication in Japan, and by KAKENHI, the Japan Society for
the Promotion of Science, grant no. 24243062.
R EFERENCES[1] J. Wolpaw and E. W. Wolpaw, Eds., Brain-Computer Interfaces: Principles and Practice. Oxford University Press, 2012.
[2] M. Chang, N. Nishikawa, Z. R. Struzik, K. Mori, S. Makino,
D. Mandic, and T. M. Rutkowski, “Comparison of P300
responses in auditory, visual and audiovisual spatial speller BCI
paradigms,” in Proceedings of the Fifth International Brain-Computer
Interface Meeting 2013. Asilomar Conference Center, PacificGrove, CA USA: Graz University of Technology Publishing House,
Austria, June 3-7, 2013, p. Article ID: 156. [Online]. Available:
http://castor.tugraz.at/doku/BCIMeeting2013/156.pdf[3] J. E. Huggins, C. Guger, B. Allison, C. W. Anderson, A. Batista,
A.-M. Brouwer, C. Brunner, R. Chavarriaga, M. Fried-Oken,
A. Gunduz, D. Gupta, A. Kübler, R. Leeb, F. Lotte, L. E.
Miller, G. Müller-Putz, T. Rutkowski, M. Tangermann, andD. E. Thompson, “Workshops of the fifth international braincomputer interface meeting: Defining the future,” Brain-Computer
Interfaces, vol. 1, no. 1, pp. 27–49, 2014. [Online]. Available:
http://www.tandfonline.com/doi/abs/10.1080/2326263X.2013.876724[4] T. M. Rutkowski and H. Mori, “Tactile and bone–conduction auditory
brain computer interface for vision and hearing impaired users,” Journalof Neuroscience Methods, p. Available online 21 April 2014, 2014.
[Online]. Available: http://dx.doi.org/10.1016/j.jneumeth.2014.04.010[5] T. M. Rutkowski, A. Cichocki, and D. P. Mandic, “Spatial auditory
paradigms for brain computer/machine interfacing,” in International
Workshop On The Principles and Applications of Spatial Hearing 2009
(IWPASH 2009) - Proceedings of the International Workshop, MiyagiZao Royal Hotel, Sendai, Japan, November 11-13, 2009, p. P5.
[6] Y. Matsumoto, N. Nishikawa, S. Makino, T. Yamada, and T. M.
Rutkowski, “Auditory steady-state response stimuli based bci
application-the optimization of the stimuli types and lengths,” pp.
1–7, 2012.
[7] G. Muller-Putz, R. Scherer, C. Neuper, and G. Pfurtscheller, “Steadystate somatosensory evoked potentials: suitable brain signals for braincomputer interfaces?” Neural Systems and Rehabilitation Engineering,IEEE Transactions on, vol. 14, no. 1, pp. 30–37, 2006.
[8] H. Mori, Y. Matsumoto, V. Kryssanov, E. Cooper, H. Ogawa, S. Makino,
Z. R. Struzik, and T. M. Rutkowski, “Multi-command tactile brain
computer interface: A feasibility study,” in Haptic and Audio Interaction
Design. Springer, 2013, pp. 50–59.

[9] H. Mori, Y. Matsumoto, S. Makino, V. Kryssanov, and T. M.
Rutkowski, “Vibrotactile stimulus frequency optimization for thehaptic BCI prototype,” in Proceedings of The 6th International
Conference on Soft Computing and Intelligent Systems, and The
13th International Symposium on Advanced Intelligent Systems, Kobe,
Japan, November 20-24, 2012, pp. 2150–2153. [Online]. Available:
http://arxiv.org/abs/1210.2942[10] D. J. Krusienski, E. W. Sellers, F. Cabestaing, S. Bayoudh, D. J.
McFarland, T. M. Vaughan, and J. R. Wolpaw, “A comparison ofclassification techniques for the P300 speller,” Journal of NeuralEngineering, vol. 3, no. 4, p. 299, 2006. [Online]. Available:
http://stacks.iop.org/1741-2552/3/i=4/a=007[11] G. Schalk and J. Mellinger, Human-Computer Interaction: Practical
Guide to Brain-Computer Interfacing with Bci2000: General-Purpose
Software for Brain-Computer Interface Research, Data Acquisition,
Stimulus Presentation, and Brain Monitoring. Springer, 2010.