Poster: Interest Detection While
Reading Newspaper Articles byUtilizing a Physiological Sensing
Wristband
Soumy Jacob
University of Kaiserslautern &German Research Center for
Artificial Intelligence (DFKI)
Kaiserslautern, Germany
Soumy.Jacob@dfki.de

Andreas Dengel
University of Kaiserslautern &German Research Center for
Artificial Intelligence (DFKI)
Kaiserslautern, Germany
Andreas.Dengel@dfki.de

Shoya Ishimaru
University of Kaiserslautern &German Research Center for
Artificial Intelligence (DFKI)
Kaiserslautern, Germany
Shoya.Ishimaru@dfki.deAbstract
In this paper, we present how physiological measures including heart rate (HR), electrodermal activity (EDA) andblood volume pulse (BVP) can be retrieved from a wristband device like an E4 wristband and further used to detect
the interest of a user during a reading task. From the dataof 13 university students on 18 newspaper articles, we haveclassified their interest level into four classes with an accuracy of 50%, and 68% with binary classification (interestingor boring). This research can be incorporated in the realtime prediction of a user’s interest while reading, for the betterment of future designs of human-document interaction.

Author Keywords
Reading; interest; heart rate; blood volume pulse; skin temperature; inter-beat interval; electrodermal activity

ACM Classification Keywords
H.5.0 [Information interfaces and presentation (e.g., HCI)]:
GeneralPermission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
Copyright held by the owner/author(s).
UbiComp/ISWC’18 Adjunct, October 8–12, 2018, Singapore, Singapore
ACM 978-1-4503-5966-5/18/10.
https://doi.org/10.1145/3267305.3267666

78Introduction
Interest in reading is built with concentration, curiosity, anddemand. It may not rise out of habit but it will help motivate
the habit and subsequently the learning process [8]. The
urge in reading, if recognized, can be used to improve thedata made available to the reader and also help in betterhuman-document interaction and the design. Predicting areader’s interest can help to make document more interactive or dynamic [1, 4]. Wearable devices like the E4 wristband 1 are unobtrusive and will assist in the real-time monitoring/gathering of physiological data. The E4 wristband
can also be used for monitoring signals during reading frompaper as well as a computer screen.

Figure 1: E4 wristband and signals

EDA raw signal
0.80.60200

4006008001000 1200 1400

EDA phasic component
0.20.00200400600800

1000 1200 1400

EDA phasic peaks
100200400600800

1000 1200 1400EDA tonic component

In this paper, we pursue to study a reader’s interest and its
association to a physiological wristband measured features
like heart rate, blood volume pulse, electrodermal activity
and arm temperature. Research questions addressed inthis study are two-fold. RQ1: What useful features can beextracted from the physiological data obtained from a wearable, RQ2: How accurately can HR, BVP, EDA, and TEMP
estimate a reader’s interest?

Blood volume pulse (BVP) is the change in volume of bloodover a given period of time. Certain emotions can trigger
the release of hormones, such as epinephrine and norepinephrine, which will increase blood flow to bring more
oxygen to the muscles. BVP can be monitored using photoplethysmography (PPG) which is a non-invasive techniquethat relies on light absorption and reflection. The signals
detected will form a wave, which represent the change inblood volume proportional to the heart rate. Adjacent local
peaks from this wave indicates heart beats and the time interval between these peaks is the inter-beat interval (IBI).
Heart rate, IBI and BVP has been associated with frustration and anxiety [5].

Background of Physiological Sensing

ApproachElectrodermal activity (EDA) refers to the autonomic changes
in the electrical properties of the skin. It is a physiologicalsignal which reliably represents the sympathetic nervous
system (SNS) and hence can be used to assess the humanemotional state. EDA is composed of a low frequency tonic
component which includes slow drifts of the baseline skinconductance level and spontaneous fluctuations in the component; and a high frequency phasic component i.e., theskin conductance response (SCR), reflects the short-time
response to the stimulus.

We utilize E4 for to measure a user’s behavior. It is used forthe acquisition of real-time physiological data with the helpof sensors designed to gather high-quality data. It has a
PPG sensor which measures BVP, an EDA sensor to measure electrical properties of the skin, an infrared thermopile
to measure skin temperature.

0.80.60200

400600800

1000 1200 1400

Figure 2: The decomposition of an
EDA signal by cvxEDA algorithm

EDA peak height and the instantaneous peak rate depict
the stress level of a person [7].

EDA has been closely associated with emotional and cognitive processing, and also used as a sensitivity index for
emotional processing and sympathetic activity [2]. Implicit
emotional responses that may occur unconsciously likethreat, anticipation, salience, novelty; can be examined using EDA. Analysis of the data by Setz et al. showed that the1

https://www.empatica.com/research/e4/

We decompose a raw EDA signal to the phasic and tonic
component as shown in Figure 2 by utilizing cvxEDA algorithm [3]. Then the following 6 features are calculated fromthe components. (1) the slope of the tonic part of the signal
for which the slope of the line of best fit was used (LinearRegression), (2) EPC - sum of all positive EDA changes, (3)Minimum peak amplitude of the phasic signal, (4) Maximum
peak amplitude of the phasic signal, (5) Mean amplitude ofthe phasic signal and (6)Number of phasic responses [6].
The Empatica developer SDK includes a sample application
which calculates IBI values from the wristband. However,

79

No.
123, 4567, 89, 1011, 12

131415, 16
1718feature
Slope of the tonic component
EPC - number ofpositive EDA changes{Min, max} peak amplitudein the phasic component
Mean amplitude ofthe phasic component
Number of phasic responses{Mean,SD} of BVP{Mean, SD} of HR
Difference in {mean, SD} ofHR amplitude during task
and baseline
SD of IBI normalized by baselineRMSSD between IBI
normalized by baseline{Mean, SD} of temperature
Mean difference oftemperture with baseline
Slope of temperature

Table 1: List of features

this was not very reliable since even the smallest of movement artifacts resulted in getting no IBI data at all. Hence,
the blood volume pulse data was used to retrieve HR andIBI. BVP data comprises of peaks called R-peaks which
can be detected by determining the region of interest foreach R-peak. A moving average is calculated so that theregion of interest will lie above this. The highest point of theretrieved region of interest is the position of the peak in theBVP signal (see Figure 3). The time interval in milliseconds
between the R-peaks is the inter beat interval and 60/IBI is
the heart rate. To further smoothen the signal, the Butterworth low-pass filter was used before the peak detection.
Features relevant to HR, IBI and BVP are extracted fromthe data. The features used are - (7, 8) mean and standard deviation of BVP, (9, 10) mean and standard deviationof HR, (11, 12) difference in mean and standard deviationof HR amplitude during task and baseline, (13) standard
deviation of IBI normalized by baseline (data recorded 5
seconds before start), (14) RMSSD - square root of themean of the square of the successive differences between
IBI. Features from skin temperature recorded by the wearable are also included namely, (15, 16) mean and standard
deviation of skin temperature, (17) difference in mean oftemperature during task and baseline, and (18) slope oftemperature.

Experiment
In order to evaluate our proposed interest detection method,
we conducted an experiment. The following section describes the experimental design and the analysis results.

Figure 3: Peak detection on BVP
signal over the moving average.

80

Experimental designThirteen university students (mean age: 25, std: 3, male: 6,female: 7, 2 of them are familiar with sensors) participatedin the experiment where each of them was asked to readeighteen newspaper articles comprising of 403 - 649 words
each (mean: 555, std: 70) with wearing E4. After readingeach document, participants answered questions regarding
their interest and objective comprehension; (1) the level ofinterest they had in the article, which was used as groundtruth (from 1 to 4, where 1 indicated ’No interest’ and 4 indicated ’High interest’), (2) one question on the article (i.e.,objective comprehension).
We followed three different approaches to separate thetrain-test data before classification. Leave-one-recordingout (LORO) uses each recording (data of each participant
on each document) as test data, the rest as training and theaverage of the accuracy in all cases together is taken as
the classification accuracy. Similar to this approach, leaveone-document-out (LODO) approach exempts the data ofa document completely from the training set and uses it fortesting. Leave-one-participant-out (LOPO) approach uses
the data from all participants except one as training.
Results and Discussion
Table 2 represents the classification accuracies using theSVM classifier with hyper parameters C: 4, gamma: 0.18
and kernel: Radial Basis Function. We also incorporated
feature reduction techniques like PCA (Principle componentanalysis) and LDA (Linear discriminant analysis), but foundno commendable improvement in the classification.
We eliminated features like - maximum phasic peak, mean
BVP and standard deviation of temperature; as SVC performs better with a reduced feature set. A backward stepwise approach was used for this purpose, where each feature was iteratively removed to achieve better classification
results. We also removed records pertaining to document
number 16 as only one of the participants had correctly
answered the objective question, which suggests that participants did not comprehend the document as expected.

An accuracy of 50% was achieved by using a 4 class SVM
classifier (Figure 4) and 68% was achieved for binary classification using cross validation and the leave one recordingout approach (LORO).
2-class

4-classLOPO5237

LODO

6346

LORO6850

Table 2: Classification accuracies
using SVM and features from E4

Conclusion
Although the accuracies were not as high as expected, thisresearch threw light on the features that could be extracted
from the physiological data from a wearable device and its
role in predicting the cognitive state of the reader. The collection of ground truth related to interest and understanding
are widely prone to human error and individual behavior,
which is also a reason for the low accuracies.
This study brought to light that physiological signal can beused to analyze cognitive states of a person with a minimum accuracy of 68%. With prolonged analysis, filtering,extraction and better data acquisition, physiological datacollected by a wristband can provide better classification
results while not only reading but aid in affective computing
during social activities, driving, and in work environments.

Acknowledgements
This work was supported by JSPS KAKENHI Grant Numbers 17K12728, 17K00276.

3. Alberto Greco, Gaetano Valenza, Antonio Lanata,
Enzo Pasquale Scilingo, and Luca Citi. 2016b. cvxEDA:A convex optimization approach to electrodermal
activity processing. IEEE Transactions on Biomedical
Engineering 63, 4 (2016), 797–804.
4. Shoya Ishimaru, Syed Saqib Bukhari, Carina Heisel,
Nicolas Großmann, Pascal Klein, Jochen Kuhn, andAndreas Dengel. 2018. Augmented Learning on
Anticipating Textbooks with Eye Tracking. In Positive
Learning in the Age of Information. Springer, 387–398.
5. Azadeh Kushki, Jillian Fairley, Satyam Merja, GillianKing, and Tom Chau. 2011. Comparison of blood
volume pulse and skin conductance responses tomental and affective stimuli at different anatomicalsites. Physiological measurement 32, 10 (2011), 1529.
6. Dominik Leiner, Andreas Fahr, and Hannah Früh.
2012. EDA positive change: A simple algorithm forelectrodermal activity to measure general audience
arousal during media exposure. Communication
Methods and Measures 6, 4 (2012), 237–250.

1. Ralf Biedert, Georg Buscher, Sven Schwarz, Jörn
Hees, and Andreas Dengel. 2010. Text 2.0. In CHI’10
Extended Abstracts on Human Factors in Computing
Systems. ACM, 4003–4008.

7. Cornelia Setz, Bert Arnrich, Johannes Schumm,
Roberto La Marca, Gerhard Tröster, and Ulrike Ehlert.
2010. Discriminating stress from cognitive load using awearable EDA device. IEEE Transactions oninformation technology in biomedicine 14, 2 (2010),410–417.

2. Alberto Greco, Antonio Lanata, Luca Citi, Nicola
Vanello, Gaetano Valenza, and Enzo Pasquale
Scilingo. 2016a. Skin admittance measurement for

8. Barry J Zimmerman. 2000. Self-efficacy: An essential
motive to learn. Contemporary educational psychology
25, 1 (2000), 82–91.

REFERENCES
Figure 4: Confusion matrix with
50% accuracy for four-class SVC.
68% accuracy for binaryclassification (1 and 2 vs. 3 and 4)

emotion recognition: A study over frequency sweep.
Electronics 5, 3 (2016), 46.

81