What’s on your mind? Mental Task Awareness Using Single
Electrode Brain Computer Interfaces
Alireza Sahami Shirazi, Mariam Hassib,
Niels Henze, Albrecht Schmidt
VIS, University of Stuttgart, Stuttgart, Germany
firstname.lastname@vis.uni-stuttgart.de
ABSTRACT

Recognizing and summarizing persons’ activities have
proven to be effective for increasing self-awareness and enable to improve habits. Reading improves one’s language
skills and periodic relaxing improves one’s health. Recognizing these activities and conveying the time spent would enable
to ensure that users read and relax for an adequate time. Most
previous attempts in activity recognition deduce mental activities by requiring expensive/bulky hardware or by monitoring
behavior from the outside. Not all mental activities can, however, be recognized from the outside. If a person is sleeping,relaxing, or intensively thinks about a problem can hardly be
differentiated by observing carried-out reactions. In contrast,
we use simple wearable off-the-shelf single electrode brain
computer interfaces. These devices have the potential to directly recognize user’s mental activities. Through a study
with 20 participants, we collect data for five representative
activities. We describe the dataset collected and derive potential features. Using a Bayesian classifier we show that reading
and relaxing can be recognized with 97% and 79% accuracy.
We discuss how sensory tasks associated with different brain
lobes can be classified using a single dry electrode BCI.
ACM Classification Keywords

H5.2 [Information interfaces and presentation]: User
Interfaces. - Graphical user interfaces.
Author Keywords

EEG, BCI, reading, general knowledge, wearable computing
INTRODUCTION

Determining the user’s activities is central for ubiquitous
computing. Being able to determine what a user is doing enables numerous use cases. The recent quantified self movement, for example, shows that there is an increasing interestin self-monitoring what we are doing. Certain mental activities, however, cannot easily be recognized from the outside.
Differentiating between sleeping, relaxing, listening to musicor thinking hard about a problem can look exactly the same
Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are notmade or distributed for profit or commercial advantage and that copies bear
this notice and the full citation on the first page. Copyrights for componentsof this work owned by others than ACM must be honored. Abstracting withcredit is permitted. To copy otherwise, or republish, to post on servers or toredistribute to lists, requires prior specific permission and/or a fee. Request
permissions from permissions@acm.org.
AH’14, March 07–09 2014, Kobe, Japan.
Copyright c 2014 ACM 978-1-4503-2761-9/14/03...$15.00.
http://dx.doi.org/10.1145/2582051.2582096

Kai Kunze
Osaka Prefecture University, Osaka, Japankunze@m.cs.osakafu-u.ac.jp
from the outside. Therefore, recognizing and monitoring cognitive processes has gained momentum as a novel source forcontextual information [8, 21].
Recognizing mental activities usually requires expensive andbulky hardware (functional magnetic resonance imaging, eye
trackers) with few notable exceptions [2, 8]. Still most of thesystems can just monitor the user for 1-2 hours due to battery
constraints and are cumbersome to wear. In this paper, we
explore to what extent an off-the-shelf, single electrode Brain
Computer Interface (BCI) system can be used to recognize
mental activities. The system itself is relatively unobtrusive,
lightweight (a head band and a mobile phone) and can be used
for long term deployments (battery life of 6-8 hours).
We are particularly interested in cognitive processes related
to learning, especially reading, because the amount we readdirectly influences the size of our vocabulary and language
skills [4]. Additionally, the more people read throughout theday the higher are their general knowledge and critical thinking skills [4, 19]. Being able to just count the minutes wedaily read would help to assess the general knowledge of aperson, as there are strong correlations between the two [19].
In addition, regular relaxation, breaks and naps are correlatedwith more effective skill acquisition and learning [6, 20]. Relaxing sufficiently often and long enough can improve ones
health, mood and fitness [16]. Thus, recognizing when a user
reads and relaxes would enable and extend applications that
help to improve users’ life.
The contributions of this paper can be summarized as follows:(1) we present a feasibility study of a wearable, unobtrusive
brain computer interface system (BCI) capable of running up
to 6-8 hours for the purpose of mental task recognition. (2)We are able to recognize reading and relaxing tasks out of 4other activities with up to 97.2% and 73.2% respectively (userdependent) and with an accuracy of 70-100 % for 8 out of 15
users (user independent). (3) We provide a dataset to otherresearchers for reproducing and building on our results.
RELATED WORK

Previously Brain Computer Interface (BCI) systems havebeen mostly used in medical and clinical research. They
are mainly used for enhancing the lives of patients with motor disabilities and brain disorders such as Alzheimer andAmyotrophic lateral sclerosis (ALS). For example, there isa semi-autonomous wheelchair that uses a BCI to retrievecertain mental signals to move the chair [7]. Furthermore,
researchers have investigated task classification using EEG
signals. Hosni et al. used the Kerin & Aunon dataset [11] andcompared three different feature extraction techniques using

Radial Basic Function and Support Vector Machine classification [10]. The best accuracy classification reported was
70%. Del R Millan et al. proposed a user-dependent neuralclassifier to recognize three mental tasks from online spontaneous EEG signals with 70% accuracy [5]. The tasks were
relax, left/right movement, cube rotation, and subtraction.
Researchers also utilized EEG signals or eye movements toclassify text comprehension, reading skills, and reading techniques. Mostow and Beck used neuro-feedback to discoverreading problems in children [12], being able to discriminate between reading easy and hard sentences. Bulling et.
al proposed a method for reading segmentation recognizing
eye movement by electrooculography [2]. In contrast to previous research, in this work we investigate the feasibility ofclassifying reading and relaxing tasks based on EEG signals
retrieved from a single electrode BCI.
With the advances in technology off-the-shelf BCIs are available which can be used in other research domains. The Epoc
BCI by Emotiv and NeuroSky devices (MindSet, MindWave,
BrainBand) are the two most popular BCI sets. The EPOC
has 14 saline sensors and two reference electrodes. Whereas
the NeuroSky BCIs have a single dry electrode. Both devices have been used in various researches. Sahami et al.
use the EPOC to annotate movies based on excitement levels [15]. NeuroPhone [3] and ThinkContacts [13] are mobile
phone BCI applications designed to help users with motor
disabilities to dial numbers without navigating to their contact list. Petersen et al.[14] used the EPOC and attempted to
distinguish among emotional responses reflected in different
scalp potentials when viewing pleasant and unpleasant pictures compared to neutral content.
DATA ACQUISITION

To evaluate the feasibility of classifying reading and relaxing using a single electrode BCI, we chose simple tasks that
contain auditory and visual stimuli as well as thinking.
Task set

Apart from reading and relaxing tasks we chose three othercommon mental tasks for classification: listening, watching amovie, and a problem solving task. For reading we had a shortstory and for problem solving we used a Sudoku game in themedium level. We recorded an audio from a popular radio station for the listening task. Finally, a short documentary video
was used for the watching task. The task set includes auditory
and visual sensory tasks which occur in different brain lobes.
Apparatus & User Study

We developed an application to collect data from the NeuroSky BrainBand device. The device is a commercial BCI
equipped with a single dry electrode placed on the subject’s
forehead. It has one reference electrode on the left ear. The
device includes a chip which filters and preprocesses the EEG
signal and transmits it via Bluetooth to the application(1 Hz).
The EEG processing protocols are not open source. As statedin the NeuroSky white papers 1 , an FFT is done on the raw
signal giving the band powers which are then scaled using a1

www.neurosky.com/AcademicPapers.aspx(accessed

21.02.14)

Figure 1. A participant playing the Sudoko task during the user study

proprietary algorithm to produce outputs which are only relative to each other.
We recruited 20 participants (8 female) with an average ageof 23.3 years (SD=2.2). The study consisted of the five aforementioned tasks. All content used during the user study wasin Arabic. Arabic was the mother-tongue of all participants
which was particularly important to ensure that no extra mental effort was exerted during the tasks. After a short introduction, the participants was asked to fill in a demographic
questionnaire. Then, they performed the five tasks one after
the other. We counterbalanced the order of the tasks to reduce
sequence effects. The study was conducted in a quite university laboratory with normal lighting conditions and minimal
noise from other electronics equipment (Figure1). Each session took approximately 35 minutes. Only three participants
had experience using a BCI device prior to the study.
DatasetWe collected following data during the user study:
eSense Values: Attention and Mediation values ranging from
1 to 100, at a sampling rate of 1 Hz. These values are determined via Neurosky proprietary algorithms. Values between
40 and 60 are considered ‘neutral’ or baseline, between 60
and 80 mean slightly elevated eSense levels, and between 80
to 100 refer to strongly elevated attention/meditation levels.
Values below 40 are interpreted as (slightly/strongly) loweredlevels. A zero eSense value means the signal cannot be calculated reliably due to background noise.
Neurosky Power Values: A series of eight 3-byte long values ranging from 0 to 224 provided at 1 Hz. These values
are: delta (0.5–2.75Hz), theta (3.5–6.75Hz), low-alpha (7.5–
9.25Hz), high-alpha (10–11.75Hz), low-beta (13–16.75Hz),
high-beta (18–29.75Hz), low-gamma (31–39.75Hz), andmid-gamma (41–49.75Hz). These values have no units,
therefore can only be interpreted by comparing them witheach other and to themselves, to consider relative quantity
and temporal fluctuations.
Blink: A one byte value ranging between 1 and 255 provided
whenever a blink is detected. The value has no unit and only
indicates the relative strength of the blink.

Raw Wave: A 16-bit value provided at 512 Hz sampling rate.
Values for the communications protocol lie in the interval
between -/+2048. Typically in EEGs, time-frequency transforms are used to change the raw signal to the frequency domain, to extract the EEG power values.
The initial analysis revealed that data collected from five participants was corrupted. Thus, the data from these five participants was removed. Further, the power values were clipped
for some seconds due to reaching the maximum value forsome of the subjects. Since this noise was minimal it wasrectified by taking the average of surrounding signals to compensate for the clipped signal at particular points.

Figure 2. The result of the user-dependent classification

TASK CLASSIFICATION

We used the annotated data collected from the 15 participants
to derive features. These features were used to train classifiers for recognizing relaxing and reading. We developed auser-independent classifier that determines the mental activity without prior training and user-dependent classifier specific for individual participants. In the following, we describe
the features derived from the data and outline the results.
Feature Set

We derived spectral and time-domain features from the collected data. In addition, we use the signals preprocessed by
the NeuroSky development kit. The features are determinedfor one second jumping windows using Matlab.
Spectral features are computed by applying a fast Fouriertransform (FFT) on the raw signal and bandpassing the delta,
theta, alpha, beta and gamma frequency bands, the average ofeach band is used as the feature. In addition, the ratios between all pairs of frequency bands are calculated. The mean
FFT value and the variance of the FFT are also used. In total, 17 spectral features are computed. Seven time-domainfeatures are extracted from the raw time-domain EEG signal.
These include the maximum positive, minimum negative andaverage amplitude of the raw signal per segment, and the Root
Mean Squared (RMS) value of the raw signal. Four features
are extracted from the NeuroSky signals: the average attention and meditation values as well as the average NeuroSky
power band values for the five frequency bands.
Classification and Results

We used the features as input to train Bayesian networks thatrecognize relaxing and reading versus the respective othertasks. The experimental results reported in the following are
obtained using WEKA [9]. All learning parameters use thedefault values in WEKA unless otherwise stated.
We trained user independent classifiers using the leave-oneout cross-validation to train a classifier and test its performance. That means we trained the classifier with data from
14 participants and evaluated the performance using the datafrom the remaining participant. The process was repeated forall participants resulting in 15 runs that were aggregated afterwards. In addition, we trained user dependent classifiers.
Four minutes of each activity were used to train the Bayesian
networks leaving 1 minute for evaluation. For both classifiers
we used the feature selection option Weka provides.

Figure 3. The result of the user-independent classification

The result revealed that the user independent classification
between reading and the other tasks was on average 68.2%
and between relaxing and others was 53.5% of all cases. The
user dependent classification determined reading vs. othertasks with 74.4% and relax vs. others with 79% on average.
Since the classification performance left room for improvement, we pairwise classified reading and relaxing with othertasks using the same classifiers. The result revealed that theclassification performance between the reading or relaxing
and all other tasks on average were more than 75% except
for the reading vs. watching movie tasks (64%). Therefore,
we excluded the movie task and repeated the classification
using the same classifiers. The result showed that the user independent classification between reading and the other tasks
excluding the watching movie was on average 68.5% and between relaxing and others was 58.2% of all cases. The user
dependent classification determined reading vs. other tasks
with 97.2% and relax vs, others with 73.2% on average. As
expected, excluding the data from the watching task increased
the performance of the classifications in total. Figures 2 and3 depict the results of user dependent and independent classification for all 15 participants.
DISCUSSION

The results show that the mental task classification has ahigher accuracy if it is preformed subject dependent. With
independent reading classification we achieve between 80 %to 100 % for 8 of the 15 participants. Although the single
electrode BCIs have a crude spatial and temporal resolution,
the results interestingly show that user-independent classification could be possible. Qualitative feedback revealed thatall participants can imagine carrying the BCI for a longer period of time during everyday life.

Furthermore, the results reveals that the EEG signals fromthe front side of the brain seem similar for the reading andwatching movie tasks. Although there is no one-to-one mapping between different mental tasks and certain brain lobes,
certain sensory tasks can be majorly associated to particularbrain lobes. For example, listening, as a auditory sensorytask, is associated with temporal lobe activity. In contrast,
listening to speech involves language understanding whichis associated with frontal lobe activity [18]. During mediating/relaxing the concentration in the relaxation process itself leads to high frontal lobe activity [1]. Brain puzzles suchas Sudoko require memory, concentration and high cognitive
load which are all functions of the frontal lobe [17]. Watching movies and reading are associated with different parts ofthe brain. While the occipital lobe is responsible for vision
which is in the rear part of the brain, the frontal lobe is responsible for interpreting. As watching a movie is mainly avisual task, the mental activity is also in the rear lobe. Since
the BrainBrand’s electrode is placed on front part of the brain,
it mainly retrieves EEG signals from the frontal lobe. Hence,
it is assumed that the BrainBrand BCI is suitable for classifying the activities mainly happen in the front side of the brain.
CONCLUSION

In this paper, we show that it is feasible to determine reading and relaxing activities using single electrode off-the-shelf
BCI systems. The findings suggest that such BCIs can beused to determine activities that mainly occur in the frontal
lobe of the brain. The distinction of reading with 97.2%
and relaxing with 73.2% in the user-dependent cases is a firststep to implement an application that can count the minutes
read/relaxed during the day. Such an application helps in assessing one’s language skills, general knowledge and learning
progress [4]. Next step is a larger study to evaluate reading
and relaxing activities during everyday living with more diverse activities.
The results also reveal that user independent classification ispossible, whereas all computer brain interfaces needed dependent training. Regarding the importance of reading as
knowledge acquisition task, this is an important insight. For
future work, we plan to investigate how to increase the recognition accuracy of the subject independent classification for
the users with low accuracy. The collected dataset is available to other researchers for other potential work.
ACKNOWLEDGMENT

This work is funded by the DFG within the SimTech Cluster of Excellence (EXC 310/1) and the EU within the Recall
project (No. 612933).
REFERENCES

1. J. Banquet. Spectral analysis of the eeg in meditation.
Electroencephalography and clinical neurophysiology,
1973.
2. A. Bulling, J. A. Ward, H. Gellersen, and G. Tröster.
Robust recognition of reading activity in transit using
wearable electrooculography. In Proc. of Pervasive,2008.
3. A. Campbell, T. Choudhury, S. Hu, H. Lu, M. Mukerjee,
M. Rabbi, and R. Raizada. Neurophone: brain-mobile
phone interface using a wireless eeg headset. In Proc. of
SIGCOMM workshop, 2010.

4. A. Cunningham and K. Stanovich. What reading does
for the mind. Journal of Direct Instruction, 2001.
5. J. del R Millan, J. Mouriño, M. Franzé, F. Cincotti,
M. Varsta, J. Heikkonen, and F. Babiloni. A local neuralclassifier for the recognition of eeg patterns associated tomental tasks. IEEE Trans. on Neural Networks, 2002.
6. N. K. Duke and P. D. Pearson. Effective practices fordeveloping reading comprehension. Journal of
education, 2008.
7. B. Graimann, B. Allison, and G. Pfurtscheller.
Brain-Computer Interfaces: Revolutionizing
Human-Computer Interaction. 2011.
8. E. Haapalainen, S. Kim, J. F. Forlizzi, and A. K. Dey.
Psycho-physiological measures for assessing cognitive
load. In Proc. of Ubicomp, 2010.
9. M. Hall and E. e. a. Frank. The weka data mining
software: an update. ACM SIGKDD Explorations
Newsletter, 2009.
10. S. Hosni, M. Gadallah, S. Bahgat, and M. AbdelWahab.
Classification of eeg signals using different feature
extraction techniques for mental-task bci. In Proc. of
ICCES, 2007.
11. Z. Keirn and J. Aunon. A new mode of communication
between man and his surroundings. IEEE Transactions
on Biomedical Engineering, 1990.
12. J. Mostow, K. Chang, and J. Nelson. Toward exploitingeeg input in a reading tutor. In Artificial Intelligence in
Education, 2011.
13. M. Perkusich, T. Rached, and A. Perkusich.
Thinkcontacts: Use your mind to dial your phone. In
Proc. of ICCE, 2011.
14. M. Petersen, C. Stahlhut, A. Stopczynski, J. Larsen, and
L. Hansen. Smartphones get emotional: mind reading
images and reconstructing the neural sources. Affective
Computing and Intelligent Interaction, 2011.
15. A. Sahami Shirazi, M. Funk, F. Pfleiderer, H. Glück, and
A. Schmidt. MediaBrain: Annotating Videos based on
Brain-Computer Interaction. 2012.
16. C. Smith, H. Hancock, J. Blake-Mortimer, and
K. Eckert. A randomised comparative trial of yoga andrelaxation to reduce stress and anxiety. Complementary
Therapies in Medicine, 2007.
17. R. Sobolewski, R. Reilly, S. Finnigan, P. Dockree,
K. O’Sullivan, and I. Robertson. Monitoring of cognitive
processes in older persons. In Proc. of NER, 2009.
18. L. Stewart, K. Von Kriegstein, J. Warren, andT. Griffiths. Music and the brain: disorders of musicallistening. Brain, 2006.
19. P. Terenzini, L. Springer, E. Pascarella, and A. Nora.
Influences affecting the development of students’ criticalthinking skills. Research in higher education, 1995.
20. P. Ward, N. J. Hodges, A. M. Williams, and J. L.
Starkes. Deliberate practice and expert performance.
Skill acquisition: Research, theory and practice, 2004.
21. Z. Ye, Y. Li, A. Fathi, Y. Han, A. Rozga, G. D. Abowd,
and J. M. Rehg. Detecting eye contact using wearable
eye-tracking glasses. In Proc. of Ubicomp, 2012.