Classification Improvement of P300 Response Based
Auditory Spatial Speller Brain–Computer Interface
Paradigm
Moonjeong Chang

Shoji Makino

Life Science Center of TARA
University of Tsukuba
Tsukuba, Ibaraki, 305-8577 Japan
Email: moonjeong@bci-lab.info

Life Science Center of TARA
University of Tsukuba
Tsukuba, Ibaraki, 305-8577 Japan
Email: maki@tara.tsukuba.ac.jp

Tomasz M. Rutkowski1,†,‡†

Life Science Center of TARA
University of Tsukuba
Tsukuba, Ibaraki, 305-8577 Japan‡RIKEN Brain Science InstituteWako-shi, Saitama, 351-0198 Japan
Email: tomek@tara.tsukuba.ac.jp

Abstract—The aim of the presented study is to provide acomprehensive test of the EEG evoked response potential (ERP)feature selection techniques for the spatial auditory BCI–speller
paradigm, which creates a novel communication option forparalyzed subjects or body–able individuals requiring a direct
brain–computer interfacing application. For rigor, the study isconducted with 16 BCI–naive healthy subjects in an experimental
setup based on five Japanese hiragana characters in an offline
processing mode. In our previous studies the spatial auditory
stimuli related P300 responses resulted with encouragingly separable target vs. non-target latencies in averaged responses, yetthat finding was not well reproduced in the online BCI single
trial based settings. We present the case study indicating that theauditory spatial unimodal paradigm classification accuracy canbe enhanced with an AUC based feature selection approach, as
far as BCI-naive healthy subjects are concerned.

I.

I NTRODUCTION

Contemporary brain–computer interface (BCI) paradigms
rely mostly on unimodal approaches [1] with still not fully
satisfactory online interfacing accuracy results. The recently
proposed solutions enhance the existing paradigms by adding
spatial stimuli variability [2], [3], [4], [5] in order to augment the brain–computer interfacing comfort or to boost the
information-transfer-rate (ITR) achieved by the users in real–time.
We enhance classification accuracy of the previously published by the authors [6], [7] simple five Japanese hiragana
characters (a,i,u,e,o) auditory spatial speller task in order toboost interfacing accuracy variability and users’ subjective
comfort. The concept is tested with 16 BCI–naive subjects
in an offline BCI processing mode. In order to do so, we
design the auditory task with synthetic vowel representations
originating from the five spatial directions using a vector
based amplitude panning (VBAP) [8] technique. Next we
propose a new evoked response potential (ERP) feature selection technique based on the area under the curve (AU C) [9]distributions separability test. We compare the results with theclassical coefficient of determination based methods [10].
The paper is organized as follows. In the next section
we introduce BCI psychophysical and EEG experimental pro1 The corresponding author.

Fig. 1.
The auditory spatial BCI–speller paradigm realized with VBAP
technique [8].

tocols. After that results of the experiments are discussed
together with the proposed ERP feature selection enhancement
technique. Discussion and future research directions concluded
the paper.
II.

METHODSIn the experiments reported in this paper, 16 healthy BCI–
naive subjects took part (mean age 21.81, standard deviation of0.75). All the experiments were performed in the Life Science
Center of TARA, University of Tsukuba, Japan. The online
EEG BCI experiments were conducted in accordance with
WMA Declaration of Helsinki - Ethical Principles for Medical
Research Involving Human Subjects. The subjects performed
the experiments with monetary gratification.
The 200 ms long spatial unimodal auditory stimuli were
presented from five distinct spatial locations. We designed a
VBAP [8] application that positions virtual sound images atspatial locations positioned at −90◦ , −45◦ , 0◦ , 45◦ , and 90◦
in respect to the front of the subject head as depicted in
Figure 1. Each subject first conducted a short psychophysicaltest with a button press response to confirm understanding of

TABLE I.

S PATIAL AUDIO BCI– SPELLER CLASSIFICATION ACCURACY
IMPROVEMENT WITH THE PROPOSED AU C– BASED ERP FEATURE
SELECTION (N aN - STANDS FOR NOT– A – NUMBER WHEN IT WAS NOTPOSSIBLE TO TRAIN THE CLASSIFIER ). T HE CLASSICAL SWLDA
CLASSIFIER WAS APPLIED IN THE SAME CONFIGURATION FOR THE TWOFEATURE SELECTION CASES . T HE TEN ERP AVERAGES METHOD [1], [10]WAS USED IN ORDER TO ENHANCE THE EEG FEATURES FROM THE
BCI– NAIVE SUBJECTS . P LEASE NOTE , THAT WHILE IN SIX CASES THE
PROPOSED ERP FEATURE EXTRACTION METHOD DID NOT IMPROVE THE
RESULTS , THERE WAS NO CASE OF A RESULT DECREASE .

300

400

500

Subject
#1
#2
#3
#4
#5
#6
#7
#8
#9
#10
#11
#12
#13
#14
#15
#16

r 2 –based accuracy
100%
40%
40%
20%
20%
60%
80%
20%
20%
40%
40%
80%
40%
0%
0%
0%

AU C–based accuracy
100%
80%
80%
20%
20%
60%
80%
80%
60%
60%
60%
80%
80%
60%
60%
20%

Improvement
0%
40%
40%
0%
0%
0%
0%
60%
40%
20%
20%
0%
40%
60%
60%
20%

0

100

200

response time [ms]

600

700

800

Psychophysical responses to auditory stimuli

a

i

u

e

o

stimulus lettersFig. 2. Results of the psychophysical experiment in a form of boxplots withaveraged response delays from the 16 subjects for each auditory spatial letter
separately. The “hiragana” spatial stimulus letters (a, i, u, e, o) are represented
by the numbers (1, 2, 3, 4, 5) in the figure.

the experimental setup. During the online BCI experiments,
the EEG signals were captured with a 16 active electrodes
EEG amplifier system g.USBamp by g.tec. The electrodes
were attached to the following head locations Cz, CPz,
POz, Pz, P1, P2, C3, C4, O1, O2, T7, T8, P3, P4, F3,
and F4, as in the 10/10 extended international system [11](see topographic plots in Figures 4 and 6). The ground andreference electrodes were attached at FCz and the left earlobe
respectively. The recorded EEG signals were processed by thein-house enhanced BCI2000 application using an SWLDA
classifier with features drawn from 0 − 600 ms ERP intervals.
The sampling frequency was set to 512 Hz, a high pass filter at
0.1 Hz, the low pass filter at 100 Hz, with an electric power line
interference notch filter set in a band 48 − 52 Hz. An inter–stimulus–interval (ISI) was set to 500 ms and each stimuli
had length of 200 ms. The subject was instructed to spell fivehiragana letters random sequences as in a classical P300 based
oddball paradigm [1]. Each target was presented ten times ina single spelling trial and the averages of ten ERPs were later
used for the classification in order to make the experimenteasier for novices.
In this paper we report the off–line postprocessing of theEEG data in order to enhance the classification results by the
new AU C–based feature extraction method.
III.

RESULTS

The averaged results of the spatial auditory psychophysicalexperiments conducted with the 16 subjects are summarized in

the Figure 2 confirming the experimental hypothesis that all
the chosen sound location stimuli had the same cognitive loads
resulting with the equal behavioral time response results (nosignificant differences as tested with the pairwise W ilcoxon–test).
The averaged EEG ERP responses from the same 16 BCI–
naive subjects are reported in Figure 3 and for a single subject
with the best accuracy boost with the proposed method (seeTable I) in Figure 5. The P300 response is clearly depictedin the target averages in the range of 400 − 500 ms incase of all subject averages. The results of r2 versus AU C
scores comparison depicted in Figure 4 and even with stronger
effect of the single subject results in Figure 6 confirm thesuperiority of the second method (AU C) for the successful
feature selection.
The resulting classification enhancement using a standard
SWLDA classifier (the same configuration for the both r2 and
AU C based features) is reported in Table I. For the majority ofsubjects performing spatial auditory BCI–speller task the AU C
based ERP features resulted in classification accuracy boosts.
Only six out of sixteen subject results remained unchanged.
None of the results suffered accuracy decrease. The single
subject #15 results are plotted in Figure 5 separately since
this has been the most difficult case resulting with the best
improvement of 60%. Figures 5 and 6 clearly show the superiorof AU C based features since they were not drawn from theregion of 400 − 600 ms which that particular subject had
somehow more active for the both targets and non–targets.
The r2 based measure unfortunately identified this region (seemiddle panel of the Figure 6) as “a potentially separable.”IV.

C ONCLUSIONSIn the paper we reported the results obtained with AU C–
based EEG ERP feature selection technique which turnedout superior comparing with the traditional r2 based method.
AU C–based EEG ERP feature extraction is the data–driven

2

r at 430 ms

Averaged targets

[µV]

Cz
CPz
POz
Pz
P1
P2
C3
C4

1.5

3.5

0.53

3

0.52

2.5

0.51

0.5

2

0

1.5

−0.5

1

0.5
0.49

0.5

0.48

Targets vs. non−targets r2 scores

x 10

−1.5
−2
−2.5
−3
0

100 200 300 400 500 600 700 800

r2

time [ms]Averaged non−targets
Cz
CPz

2

Cz
CPz
POz
Pz
P1
P2
C3
C4
O1
O2
T7
T8
P3
P4
F3
F4

1.5

POz
Pz
P1
P2
C3
C4

3.5
3
2.5
2
1.5
1
0.5

Targets vs. non−targets AUC scores

0.5
0
−0.5
−1
−1.5
−2
−2.5
−3
0

100 200 300 400 500 600 700 800

time [ms]

1

O1
O2
T7
T8
P3
P4
F3
F4

−3

4

0

AUC

[µV]

0.54

2

−1

T7
T8
P3
P4
F3
F4

AUC at 430 ms

4

1

O1
O2

−3

x 10

Cz
CPz
POz
Pz
P1
P2
C3
C4
O1
O2
T7
T8
P3
P4
F3
F4

0.54
0.53
0.52
0.51
0.5
0.49
0.48
0

100 200 300 400 500 600 700 800

100 200 300 400 500 600 700 800

time [ms]time [ms]Fig. 3. EEG ERP averaged responses of the 16 BCI–naive subjects to targets
in the upper panel and non–targets in the lower one. P300 response is easy
to identify in the range of 300 − 600 ms. The associated head topographic
plots, r2 and AU C results are depicted in Figure 4.

Fig. 4. The results of the 16 subjects averaged r2 versus AU C based ERP
feature selection in form of respective topographic plots in the top panels.
Time series of r2 (middle panel) and AU C (bottom panel) scores are also
depicted.

approach without assumptions on data distribution, whichapparently results with longer latencies leading to improved
classification results.

ACKNOWLEDGMENTS

The obtained results allowed for the boost up to 60% of theoffline BCI mode classification results in the five–commands
spatial auditory paradigm as reported in the Table I.
The presented preliminary, yet encouraging results, call for
more research on spatial auditory BCI paradigms. The next
research steps will include the spatial auditory stimulus optimization for handicapped or bedridden subjects, who cannot
utilize the fully surround acoustic environment.
We plan to continue this research in order to apply themethod in online BCI application for those patients suffering amyotrophic–lateral–sclerosis (ALS), also known as
Lou Gehrig’s disease, or totally–locked–in–syndrome (TLS)
to create a new communication possibility for such users inneed.

This research was supported in part by the Strategic Information and Communications R&D Promotion Programme no.
121803027 of The Ministry of Internal Affairs and Communication in Japan, and by KAKENHI, the Japan Society for the
Promotion of Science grant no. 12010738.
We also acknowledge the technical support from
YAMAHA Sound & IT Development Division in Hamamatsu,
Japan.
R EFERENCES[1]

J. Wolpaw and E. W. Wolpaw, Eds., Brain-Computer Interfaces: Principles and Practice. Oxford University Press, 2012.
[2] S. Halder, M. Rea, R. Andreoni, F. Nijboer, E. Hammer, S. Kleih,
N. Birbaumer, and A. Kübler, “An auditory oddball brain–computer interface for binary choices,” Clinical Neurophysiology,
vol. 121, no. 4, pp. 516 – 523, 2010. [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S1388245709007512

2

r at 461 ms

Averaged targets
2

[µV]

Cz
CPz
POz
Pz
P1
P2
C3
C4

0

0.5
0.48

2

0.46
0.44

−3
−4

Targets vs. non−targets r2 scores−5−6
100 200 300 400 500 600 700 800

r2

Averaged non−targets
2

Cz
CPz

1

POz
Pz
P1
P2
C3
C4

Cz
CPz
POz
Pz
P1
P2
C3
C4
O1
O2
T7
T8
P3
P4
F3
F4

−3

x 10
16
14
12
10
8
6
4
2

0

100 200 300 400 500 600 700 800

time [ms]0Targets vs. non−targets AUC scores
−1
−2

O1
O2
T7
T8
P3
P4
F3
F4

−3

AUC

[µV]

10

4

time [ms]

−4
−5
−6
0

Fig. 5. EEG ERP averaged responses of the subject #15 whose target (upper
panel) and non–target (lower panel) responses did not differ significantly as
if in this case the user fired P300 response for each stimuli. Fortunately, thepropose AU C–based feature selection allowed for 60% classification boost(see Table I). The associated head topographic plots, r2 and AU C results are
depicted in Figure 6.

G. Muller-Putz, R. Scherer, C. Neuper, and G. Pfurtscheller, “Steadystate somatosensory evoked potentials: suitable brain signals for braincomputer interfaces?” Neural Systems and Rehabilitation Engineering,IEEE Transactions on, vol. 14, no. 1, pp. 30–37, March 2006.

[5]

M. van der Waal, M. Severens, J. Geuze, and P. Desain, “Introducing
the tactile speller: an ERP-based brain–computer interface forcommunication,” Journal of Neural Engineering, vol. 9, no. 4,
p. 045002, 2012. [Online]. Available: http://stacks.iop.org/17412552/9/i=4/a=045002
M. Chang, N. Nishikawa, Z. Cai, S. Makino, and T. M. Rutkowski,“Psychophysical responses comparison in spatial auditory, visual,and audiovisual BCI-spelling paradigms,” in Proceedings of The 6thInternational Conference on Soft Computing and Intelligent Systems,
and The 13th International Symposium on Advanced Intelligent
Systems, Kobe, Japan, November 20-24, 2012, pp. 2154–2157.
[Online]. Available: http://arxiv.org/abs/1210.2959
M. Chang, N. Nishikawa, Z. R. Struzik, K. Mori, S. Makino, D. Mandic,

0.54
0.52
0.5
0.48
0.46
0.44
100 200 300 400 500 600 700 800

time [ms]Fig. 6. The comparison of the single subject #15 results of r2 versus AU C–
based ERP feature selection in form of respective topographic plots in the top
panels. Time series of r2 (middle panel) and AU C (bottom panel) scores
are also depicted. In this case the difference between r2 and AU C measures
is significant for the 400 − 500 ms latencies with the troublesome for thissubject P300 response fired for the both targets and non-targets (compare theboth panels in Figure 5). The AU C based features were not drawn from thislatency range.

M. Schreuder, B. Blankertz, and M. Tangermann, “A new auditory
multi-class brain-computer interface paradigm: Spatial hearing as an
informative cue,” PLoS ONE, vol. 5, no. 4, p. e9813, 04 2010.

[4]

Cz
CPz
POz
Pz
P1
P2
C3
C4
O1
O2
T7
T8
P3
P4
F3
F4
0

100 200 300 400 500 600 700 800

time [ms]

[7]

0.52

6

−2

0

[6]

0.54

12

8

−1

T7
T8
P3
P4
F3
F4

[3]

AUC at 166 ms

14

1

O1
O2

−3

x 10
16

[8]

[9]
[10]
[11]

and T. M. Rutkowski, “Comparison of P300 responses in auditory,visual and audiovisual spatial speller BCI paradigms,” in Proceedingsof the Fifth International Brain-Computer Interface Meeting 2013.
Asilomar Conference Center, Pacific Grove, CA USA: Graz Universityof Technology Publishing House, Austria, June 3-7, 2013, p. Article
ID: 156.
V. Pulkki, “Virtual sound source positioning using vector base amplitude
panning,” Journal of Audio Engineering Society, vol. 45, no. 6, pp. 456–
466, 1997.
S. Theodoridis and K. Koutroumbas, Pattern Recognition, fourh ed.
Acedemic Press, 2009.
G. Schalk and J. Mellinger, A Practical Guide to Brain–Computer
Interfacing with BCI2000. Springer-Verlag London Limited, 2010.
V. Jurcak, D. Tsuzuki, and I. Dan, “10/20, 10/10, and10/5 systems revisited: Their validity as relative headsurface-based positioning systems,” NeuroImage, vol. 34,
no. 4, pp. 1600 – 1611, 2007. [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S1053811906009724