Multi–command Chest Tactile Brain Computer
Interface for Small Vehicle Robot Navigation
Hiromu Mori1 , Shoji Makino1 , and Tomasz M. Rutkowski1,21Life Science Center of TARA, University of Tsukuba, Tsukuba, Japan
2
RIKEN Brain Science Institute, Wako-shi, Japan
tomek@tara.tsukuba.ac.jp

Abstract. The presented study explores the extent to which tactile
stimuli delivered to ﬁve chest positions of a healthy user can serve asa platform for a brain computer interface (BCI) that could be used in aninteractive application such as robotic vehicle operation. The ﬁve chest
locations are used to evoke tactile brain potential responses, thus deﬁning a tactile brain computer interface (tBCI). Experimental results with
ﬁve subjects performing online tBCI provide a validation of the chest location tBCI paradigm, while the feasibility of the concept is illuminatedthrough information-transfer rates. Additionally an oﬄine classiﬁcation
improvement with a linear SVM classiﬁer is presented through the casestudy.
Keywords: tactile BCI, P300, robotic vehicle interface, EEG, neurotechnology.

1Introduction

Contemporary BCIs are typically based on mental visual and motor imagery
paradigms, which require extensive user training and good eyesight from theusers [21]. Recently alternative solutions have been proposed to make use of spatial auditory [15,3,17] or tactile (somatosensory) modalities [12,1,11,20,10,7] toenhance brain-computer interface comfort and increase the information transfer rate (ITR) achieved by users. The concept reported in this paper further
extends the previously reported by the authors in [10] brain somatosensory (tactile) channel to allow targeting of the tactile sensory domain for the operationof robotic equipment such as personal vehicles, life support systems, etc. The
rationale behind the use of the tactile channel is that it is usually far less loaded
than auditory or even visual channels in interfacing applications.
The ﬁrst report [12] of the successful employment of steady-state somatosensory responses to create a BCI targeted a low frequency vibrotactile stimulus
in the range of 20 − 31 Hz to evoke the subjects’ attentional modulation, which
was then used to deﬁne interfacing commands. A more recent report [20] proposed using a Braille stimulator with 100 ms static push stimulus delivered toeach of six ﬁngers to evoke a somatosensory evoked potential (SEP) response
and the following P300. The P300 response is a positive electroencephalogram
K. Imamura et al. (Eds.): BHI 2013, LNAI 8211, pp. 469–478, 2013.
c Springer International Publishing Switzerland 2013


470

H. Mori, S. Makino, and T.M. Rutkowski

event-related potential (ERP) deﬂection starting at around 300 ms and lastingfor 200 − 300 ms after an expected stimulus in a random series of distractors (theso-called oddball EEG experimental paradigm) [13]. Examples of averaged P300
response are depicted with red lines with standard errors in Figures 2, 3, 4, and 5.
The P300 responses are commonly used in BCI approaches and are considered tobe the most reliable ERPs [16,21] with even beginner subjects. The results in [20]indicated that the experiments achieved information transfer rates of 7.8 bit/min
on average and 27 bit/min for the best subject. A very recent report [7] additionally conﬁrmed superiority of the tactile BCI (tBCI) in comparison with visualand auditory modalities tested with a locked–in syndrome (LIS) subject [14].
This paper reports improvement of our previously reported ﬁnger stimulustBCI [10] based on P300 responses evoked by tactile stimulus delivered via vibrotactile transducers attached to ﬁve positions on the subject’s chest this time.
The proposal is similar to the previously reported waist positions based tBCIreported in [1] with a diﬀerence that we propose to use the chest area simulation
which simpliﬁes a vehicular robot operation in comparison to our previous handand head-stimulus–based tBCI solutions reported in [10,9].
The rest of the paper is organized as follows. The next section introduces
the materials and methods used in the study. It also outlines the experiments
conducted. The results obtained in electroencephalogram online and oﬄine experiments with ﬁve BCI subjects are then discussed. Finally, conclusions areformulated and directions for future research are outlined.

2

Materials and Methods

Five volunteer male BCI subjects participated in the experiments. The subjects’mean age was 26, with a standard deviation of 9.5. All the experiments were
performed at the Life Science Center of TARA, University of Tsukuba, Japan.
The online (real-time) EEG tBCI paradigm experiments were conducted in accordance with the WMA Declaration of Helsinki - Ethical Principles for Medical
Research Involving Human Subjects.
2.1

Tactile Stimulus Generation

The tactile stimulus was delivered as sinusoidal wave generated by a portablecomputer with MAX/MSP software [4]. The stimuli were delivered via ﬁve channel outputs of an external digital-to-analog signal converter RME Fireface UCX
coupled with the two acoustic YAMAHA P4050 power ampliﬁers (four acoustic
frequency channels each). The stimuli were delivered to the subjects’ chest locations via the tactile transducers HiWave HIAX25C10-8/HS operating in the
acoustic frequency spectrum of 100 − 20, 000 Hz, as depicted in Figure 1. Eachtransducer in the experiments was set to emit a sinusoidal wave at 200 Hz tomatch the transducer’s resonance frequency. Tactile impulses were designed tostimulate the Pacini endings (fast-adapting type II aﬀerent type tactile sensory
innervation receptors) which are the large receptive ﬁeld mechanoreceptors inMulti–command Chest Tactile Brain Computer Interface

471

Fig. 1. Subject wearing EEG cap with 16 active electrodes attached to the g.USBampampliﬁer. The ﬁve vibrotactile transducers are attached to an elastic belt on the subject
chest. Small vehicle robot by LEGO MINDSTROMS, operated via tBCI application
developed by the authors, is placed on the ﬂoor in front of the subject in the picture.

deeper layers of human skin [5]. The training instructions were presented visually by means of the BCI2000 program with numbers 1 − 5 representing robot
movement directions (see Table 2) communicated via vibrotactile transducers
attached to the subject’s chest (see Figure 1).
2.2

EEG tBCI Experiment

EEG signals were captured with an EEG ampliﬁer system g.USBamp by g.tec
Medical Engineering GmbH, Austria, using 16 active electrodes. The electrodes
were attached to the head locations: Cz, Pz, P3, P4, C3, C4, CP5, CP6, P1,
P2, POz, C1, C2, FC1, FC2, and FCz, as in the 10/10 extended international
system [6] (see the topographic plot in the top panel of Figure 2). The groundelectrode was attached to FPz position and reference to the left earlobe respectively. No electromagnetic interference was observed from the vibrotactile
transducers operating in higher frequencies comparing to the EEG frequency
spectrum. Details of the EEG experimental protocol are summarized in Table 1.
The captured EEG signals were processed online by BCI2000-based application [16], using a stepwise linear discriminant analysis (SWLDA) classiﬁer [8,18]
with features drawn from the 0 − 650 ms ERP intervals. Additionally in oﬄine
mode the classiﬁcation accuracy was compared and improved using linear SVM
classiﬁer [2]. The EEG recording sampling rate was set at 512 Hz, the high pass

472

H. Mori, S. Makino, and T.M. Rutkowski
Table 1. Conditions of the EEG experiment

Number of subjects
Tactile stimulus length
Stimulus frequency
Inter-stimulus-interval (ISI)
EEG recording system
Number of the EEG channels
EEG electrode positions

5
100 ms200 Hz
400 msg.USBamp active EEG electrodes system.
16
Cz, Pz, P3, P4, C3, C4, CP5, CP6, P1,
P2, POz, C1, C2, FC1, FC2, FCz
Reference and ground electrodes
earlobe and FPz
Stimulus generation
5 HIAX25C10-8 transducers
Number of trials used by SWLDA (SVM) 7 (1)Table 2. Interactive vehicular robot driving application commands encoded with thechest position numbers.
Chest position number
1
2
3
4
5

Commandgo left (−90◦ )go straight–left (−45◦ )go straight (0◦ )go straight–right (45◦ )go right (90◦ )ﬁlter at 0.1 Hz, and the low pass ﬁlter at 60 Hz. The ISI was 400 ms, and eachstimulus duration was 100 ms. The subjects were instructed to spell out the number sequences (corresponding to the interactive robotic application commands
shown in Table 2) communicated by the transducers in each session. Each target
was presented seven times in a single command trial. Each subject performed
three experimental sessions (randomized 35 targets and 140 non−targets each),which were later averaged for the online SWLDA classiﬁer case or treated as single trial (only the ﬁrst ERP was used from a sequence of seven) for linear SVM,
as discussed in a next section. The ﬁrst online tBCI session was a subject practice, the second was used for training the classiﬁers, while the last experimental
session was used for testing interfacing accuracy.

3

Results

This section discusses the results obtained in the EEG online and oﬄine data processing experiments, which are summarized in in Table 3 and Figures 2, 3, 4, and 5.
All the participating in the study tBCI subjects scored well above the chancelevel of 20%, reaching an ITR in the range from 0.64 bit/min to 5.14 bit/min
in case of the online BCI experiment with SWLDA classiﬁer, which may beMulti–command Chest Tactile Brain Computer Interface!  



μ

   

473

"
"








"




"




ï
ï








!ï  

μ





"
"








"




"




ï
ï










 !ï    







"
"








"




"

















Fig. 2. All the ﬁve subjects grand mean averaged results of the chest stimulation EEG
experiment. The left panel presents the head topographic plot of the target versus
non − target area under the curve (AUC), a measure commonly used in machinelearning intra–class discriminative analysis. (AU C > 0.5 is usually assumed to be
conﬁrmation of feature separability [19]). The top right panel presents the largest
diﬀerence as obtained from the data displayed in the bottom panel. The topographic
plot also depicts the electrode positions. The fact that all the electrodes received similar
AUC values (red) supports the initial electrode placement. The second panel from the
top presents averaged SEP responses to the target stimuli (note the clear P300 responsein the range of 350 − 600 ms). The third panel presents averaged SEP responses to thenon − target stimuli (no P300 observed). Finally, the bottom panel presents the AUC
of target versus non − target responses (again, P300 could easily be identiﬁed).

474

H. Mori, S. Makino, and T.M. Rutkowski

Fig. 3. All ﬁve subjects grand mean averaged results for each electrode plotted separately. The red and blue lines depict targets and non–targets respectively together withstandard error bars. The P300 related responses could be observed in the 350 − 650 ms
latencies.

Multi–command Chest Tactile Brain Computer Interface

475

Fig. 4. Averaged results for each electrode separately of the subject #3 for whom P 300
response was dominating leading for the best latency used in classiﬁcation. The redand blue lines depict targets and non–targets respectively together with standard error
bars.

Fig. 5. Averaged results for each electrode separately of the subject #2 for whom
N 200 response was signiﬁcant leading for improved early latency used in classiﬁcation.
Observe the very short latency around 200ms where the standard error bars don’t
overlap. The red and blue lines depict targets and non–targets respectively together
with standard error bars.

476

H. Mori, S. Makino, and T.M. Rutkowski

Table 3. The chest positions stimulation EEG experiment accuracy and ITR scores.
The theoretical chance level was 20%. For the SWLDA classiﬁer, features were derived
from the averages of the seven ERPs of all the subjects. In case of the linear SVM
classiﬁer only single trials (sequences) were used. Observe the increase in ITR withsingle trial classiﬁcation in case of the linear SVM classiﬁer utilization.
SWLDA classiﬁer [8,18]
Subject Number of averaged trials
#1
7
#2
7
#3
7
#4
7
#5
7

Maximum accuracy
40%
60%
40%
80%
60%

ITR
0.64 bit/min
2.36 bit/min
0.64 bit/min
5.14 bit/min
2.36 bit/min

Oﬄine linear SVM classiﬁer [2] based improvement
Subject Number of averaged trials
#1
1
#2
1
#3
1
#4
1
#5
1

Maximum accuracy
100%
60%
40%
80%
60%

ITR
69.65 bit/min
16.53 bit/min
4.53 bit/min
36.00 bit/min
16.53 bit/min

considered to be a good outcome for online vehicular robot driving experiments.
The ITR was calculated as follows [17]:IT R = V · R
R = log2 N + P · log2 P + (1 − P ) · log2



1−P
N −1

(1)


,

(2)

where R stands for the number of bits/selection; N is the number of classes (5in this study); P is the classiﬁer accuracy (see Table 3); and V is the classiﬁcation speed in selections/minute (4.3 selections/minute for this study in caseof averaging of seven responses in online SWLDA case or 30 selections/minutefor the single trial based linear SVM classiﬁer). The maximum ITR possible for
the BCI subjects to achieve in the settings presented were 5.14 bit/min and
69.65 bit/min, for seven averaged and single trial cases respectively.

4

Conclusions

This case study demonstrated results obtained with a novel ﬁve–commands andchest locations based tBCI paradigm developed and used in experiments withﬁve “body–able” subjects. The proposed interface could be used for a real–time
operation of the robotic vehicle. The experiment results obtained in this study
conﬁrmed the validity of the chest tBCI for interactive applications and thepossibility to further improve the results with utilization of the single trial based
linear SVM classiﬁer.

Multi–command Chest Tactile Brain Computer Interface

477

The EEG experiment with the paradigm has conﬁrmed that tactile stimuli canbe used to operate robotic devices with ﬁve commands and with the interfacing
rate ranging from 0.64 bit/min to 5.14 bit/min for online case using SWLDA
and 4.53 bit/min to 69.95 bit/min in the oﬄine post processing case with linear
SVM classiﬁer respectively.
The results presented oﬀer a step forward in the development of novel neurotechnology applications. Due to the still not very high interfacing rate achievedin online BCI case among the subjects, the current paradigm would obviously
need improvements and modiﬁcations to implement also online the proposed andtested oﬄine linear SVM classiﬁer based processing. These needs determine themajor lines of study for future research. However, even in its current form, theproposed tBCI can be regarded as a practical solution for LIS patients (locked
into their own bodies despite often intact cognitive functioning), who cannot use
vision or auditory based interfaces due to sensory or other disabilities.
We plan to continue this line of the tactile BCI research in order to further
optimize the signal processing and machine learning (classiﬁcation) methods.
Next we will test the paradigm with the LIS patients in need for BCI technology.
Acknowledgments. This research was supported in part by the Strategic Information and Communications R&D Promotion Program no. 121803027 of The
Ministry of Internal Aﬀairs and Communication in Japan, and by KAKENHI,
the Japan Society for the Promotion of Science, grant no. 12010738. We also
acknowledge the technical support of YAMAHA Sound & IT Development Division in Hamamatsu, Japan.
Author contributions: HM, TMR: Performed the EEG experiments and analyzed the data. TMR: Conceived the concept of the spatial tactile BCI anddesigned the EEG experiments. SM: Supported the project. HM, TMR: Wrotethe paper.

References
1. Brouwer, A.M., Van Erp, J.B.F.: A tactile P300 brain-computer interface. Frontiersin Neuroscience 4(19) (2010)
2. Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: LIBLINEAR: A library for large linear classiﬁcation. Journal of Machine Learning Research 9, 1871–
1874 (2008)
3. Halder, S., Rea, M., Andreoni, R., Nijboer, F., Hammer, E., Kleih, S., Birbaumer,N., Kübler, A.: An auditory oddball brain–computer interface for binary choices.
Clinical Neurophysiology 121(4), 516–523 (2010)
4. Max 6 (2012), http://cycling74.com/
5. Johansson, R.S., Flanagan, J.R.: Coding and use of tactile signals from the ﬁngertips in object manipulation tasks. Nature Reviews Neuroscience 10(5), 345–359(2009)
6. Jurcak, V., Tsuzuki, D., Dan, I.: 10/20, 10/10, and 10/5 systems revisited: Their validity as relative head-surface-based positioning systems. NeuroImage 34(4), 1600–
1611 (2007)

478

H. Mori, S. Makino, and T.M. Rutkowski

7. Kaufmann, T., Holz, E.M., Kübler, A.: Comparison of tactile, auditory and visual
modality for brain-computer interface use: A case study with a patient in thelocked-in state. Frontiers in Neuroscience 7(129) (2013)
8. Krusienski, D.J., Sellers, E.W., Cabestaing, F., Bayoudh, S., McFarland, D.J.,
Vaughan, T.M., Wolpaw, J.R.: A comparison of classiﬁcation techniques for the
P300 speller. Journal of Neural Engineering 3(4), 299 (2006)
9. Mori, H., Matsumoto, Y., Struzik, Z.R., Mori, K., Makino, S., Mandic, D.,
Rutkowski, T.M.: Multi-command tactile and auditory brain computer interface based on head position stimulation. In: Proceedings of the Fifth International Brain-Computer Interface Meeting 2013, Asilomar Conference Center, Paciﬁc Grove, CA USA, June 3-7. Graz University of Technology Publishing House,
Austria (2013), Article ID: 095
10. Mori, H., Matsumoto, Y., Kryssanov, V., Cooper, E., Ogawa, H., Makino, S.,
Struzik, Z., Rutkowski, T.M.: Multi-command tactile brain computer interface:
A feasibility study. In: Oakley, I., Brewster, S. (eds.) HAID 2013. LNCS, vol. 7989,
pp. 50–59. Springer, Heidelberg (2013)
11. Mori, H., Matsumoto, Y., Makino, S., Kryssanov, V., Rutkowski, T.M.: Vibrotactile stimulus frequency optimization for the haptic BCI prototype. In: Proceedingsof the 6th International Conference on Soft Computing and Intelligent Systems,
and the 13th International Symposium on Advanced Intelligent Systems, Kobe,
Japan, November 20-24, pp. 2150–2153 (2012)
12. Muller-Putz, G., Scherer, R., Neuper, C., Pfurtscheller, G.: Steady-state somatosensory evoked potentials: suitable brain signals for brain-computer interfaces? IEEE Transactions on Neural Systems and Rehabilitation Engineering 14(1),30–37 (2006)
13. Niedermeyer, E., Da Silva, F.L. (eds.): Electroencephalography: Basic Principles,
Clinical Applications, and Related Fields, 5th edn. Lippincott Williams & Wilkins(2004)
14. Plum, F., Posner, J.B.: The Diagnosis of Stupor and Coma. FA Davis, Philadelphia(1966)
15. Rutkowski, T.M., Cichocki, A., Mandic, D.P.: Spatial auditory paradigms for brain
computer/machine interfacing. In: Proceedings of the International Workshop International Workshop on the Principles and Applications of Spatial Hearing 2009
(IWPASH 2009), Miyagi-Zao Royal Hotel, Sendai, November 11-13, p. 5 (2009)16. Schalk, G., Mellinger, J.: A Practical Guide to Brain–Computer Interfacing withBCI2000. Springer-Verlag London Limited (2010)17. Schreuder, M., Blankertz, B., Tangermann, M.: A new auditory multi-classbrain-computer interface paradigm: Spatial hearing as an informative cue. PLoS
ONE 5(4), e9813 (2010)18. Stocks, C.R.M.: Py3GUI (2011), https://github.com/collinstocks/py3gui
19. Theodoridis, S., Koutroumbas, K.: Pattern Recognition, 4th edn. Acedemic Press(2009)
20. van der Waal, M., Severens, M., Geuze, J., Desain, P.: Introducing the tactile
speller: an ERP-based brain–computer interface for communication. Journal of
Neural Engineering 9(4), 045002 (2012)
21. Wolpaw, J., Wolpaw, E.W. (eds.): Brain-Computer Interfaces: Principles and Practice. Oxford University Press (2012)