Assessing Cognitive Workload on Printed and Electronic
Media using Eye-Tracker and EDA Wristband
Iuliia Brishtel1,2 , Shoya Ishimaru1,2,3 ,Olivier Augereau3 , Koichi Kise3 , Andreas Dengel1,2,3
1 German Research Center for Artificial Intelligence (DFKI);
2 TU Kaiserslautern; 3 Osaka Prefecture University{Iuliia.Brishtel, Shoya.Ishimaru, Andreas.Dengel}@dfki.de, {augereau@m., kise@}cs.osakafu-u.ac.jp
ABSTRACTWith the expansion of e-learning platforms, we receive a great
opportunity to learn and study just using an electronic device.
In this paper, we measured the differences in information
processing on screen and paper with 18 participants using aneye-tracker and an EDA wristband. Our findings show that themedia type has a significant influence on cognitive workload
and understandability of the content. The results of this work
are of vital importance for the design of new intelligent userinterfaces and reveal the necessity to take mental processes ofusers more into account.
ACM Classification Keywords

H.5 Information interfaces and presentation (e.g., HCI): H.5.2
User Interfaces, Interaction Style.
Author Keywords

E-learning; Eye-Tracking; Reading; Information Processing;Electrodermal Activity; Cognitive Workload; User-Interface
INTRODUCTION

According to “Global Market Insights”, market for Learning
Management System (LMS) grows every year at a 5% and is
expected to reach approximately 240 billion USD by 2030.
In order to provide appropriate means for knowledge transfer,the design of e-learning environment plays the key role ineducational process [1].
Previous research, however, pointed out considerable differences in information processing on paper and electronic surfaces [2]. The results show significant advantages for learningprocesses on printed media compared to its digital counterpart.
For the user interface design this fact means a demand fordeeper understanding of more specific cognitive processes byusers on the one hand and implementation of new evaluation
methods to access these processes on the other hand.
Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).

IUI’18 Companion March 7–11, 2018, Tokyo, Japan© 2018 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-5571-1/18/03. . . $15.00
DOI: https://doi.org/10.1145/3180308.3180354

Figure 1. Procedural steps of our study. After calibrating an eye-tracker,participants read documents on screen and paper. We avoided ordereffects of the two tasks by dividing participants into subgroups.

In this study, we assessed cognitive workload of participants
and understandability by using four different measures: Theresult of multiple choice tests, average pupil size and fixation duration, which were found to be reliable indicators ofcognitive workload [3] and tonic component of electrodermal
activity (EDA) which application in this research area is quite
new. In summary, we could obtain significant differences bycomparing these four variables in paper and screen conditions.
EXPERIMENTAL DESIGN

Eighteen computer science students from France, Japan, Ireland and Italy with intermediate to fluent English level and anage between 21 and 27 years participated in our study. Four ofthem used contact lenses to correct vision. For participation,
they received a compensation in value of 1,000 JPY.
Two types of sensors were used, to measure cognitive workload: a wristband E4 and a wearable eye-tracker. The wristband E4 with recording rate of 4 Hz was placed on a nondominant hand of participants and switched on at the begin ofeach reading task. The binocular Pupil Labs with recording
rate of 120 Hz collected fixation duration and and average
pupil size. For the reading task on screen, we used a 15-inch
retina display. Lightness both in the room and on the screen
was kept in the same state. The standardized distance between
the used media and participants was 30 cm.
For the experiment, two passages with six related multiple
choice questions were taken from two scientific texts withthe same difficulty level, length and font style. The orderof passages was randomized between participants. We ran-Table 1. Relative changes of variables in screen condition comparing to paper condition

Variables
Average Pupil DiameterFixation Duration
EDA (Tonic Component)

The Paper First Group
Relative Difference STD Sig.
+10.71%
0.52.01+11.64%
1.53.01+73.38%23.18 .01

domly divided participants into two groups (nine participantsper group) to avoid any order effects of the used media asfollowing. Participants in the paper first group started with aprinted document and after reading solved a multiple-choicetest presented on paper (paper condition). Then, the second
document with subsequent multiple choice test was presented
on the screen (screen condition). The screen first group did
the same procedure in the reversed order as shown in Figure 1. The instruction was to read documents as quick and asattentive as possible. The time limit for each document was 7
minutes and 30 seconds and no time limit for solving tests.
Multiple Choice Test for Understandability Measurement

We combined all answers from paper and screen conditions
into two groups. Consequently, group means were calculated
and analyzed with the Wilcoxon-Mann-Whitney since the data
did not satisfy requirements of t-test.
Average Pupil Size and Fixation Duration Processing

Average pupil size and fixation duration were extracted byPupil Labs software. In the preprocessing stage, raw data
was filtered by 10 Hz low-pass filter and then controlled foroutliers. In the next step, we calculated individual means forboth variables in paper and screen conditions and run t-test
on individual level in both groups. Finally, average pupil sizeand fixation duration from paper conditions were taken as abaseline for calculation of relative changes in these variables
in related screen conditions.
EDA Processing

EDA relates to the sympathetic nerve system (SNS) and increase in physical, emotional or cognitive state can be obtainedin rising of EDA signal. Tonic component is one of electrodermal measures which activity is associated with internal
information processing [4]. It was processed by the method
proposed by Greco et al. [5]. The data was filtered by 2 Hz lowpass forward-backward digital filter and then tonic component
was extracted. In the next step, we inspected data for outliers.
Then, individual mean of tonic component for each condition
was calculated and analyzed by t-test. Finally, EDA signal
in paper conditions was taken as a baseline for calculation ofrelative changes in EDA signal in screen conditions.
RESULTSThe rate of correctly given answers in the multiple choice test
after reading print documents was 72.2% while after reading
documents on screen the results were at 13.9% worse (p <.05).
Table 1 shows the relative changes of the variables and pvalues of t- test in the screen condition versus paper condition
in both groups. In both groups the average fixation duration
while reading on screen was significantly higher compared
to reading on the paper. The same significantly increase wasobtained with the pupil diameter.

The Screen First Group
Relative Difference
STD+10.91%0.60+11.07%1.60+74.49 / -32.90%
7.78/18.10

Sig.
.01
.01.01In the paper first group, tonic component by reading on screen
significantly increased in average to 73.38% compared to thepaper condition. In the screen first group, two tendencies inchanges of magnitudes in tonic component were obtained:by four participants, the magnitude of tonic component byreading on screen increased in average to 74.49%, while forfive participants this magnitude decreased in average at 32.90%
comparing to the reading on paper.
DISCUSSION

The results of the multiple choice test show significantly better
performance in test solving after reading the documents on
paper than on screen, which is consistent with a number ofseveral studies [2]. The findings in the pupil diameter sizeand fixation duration in our study are consistent with previous
studies: in response to rising cognitive workload pupil diameter and fixation duration significantly increase [3]. This resultis interesting in the way of natural response of the pupil on
the light from computer screen: since the pupil size decreases
in response to light source we obtained here an opposite effect comparing it with response on paper. Another interestingfinding was done in the screen first group where some participants had significantly higher level of tonic component while
reading on paper. This could be explained by the order effect:
fatigue from the screen conditions could trigger an increase of
EDA magnitude by reading on paper.
CONCLUSION

We present two contributions in the field of designing intelligent user interfaces. First, the results of this study show
a significant difference in cognitive workload by the same
information processing on screen or paper. Therefore, thesefindings should be considered more carefully in user interface
design in order to make e-learning environment less demanding for users. Secondly, our experiment shows that there
are new opportunities to assess mental workload using nonexpensive, simple and pervasive devices like EDA wristband.
REFERENCES
1. B. Faghih, M.R. Azadehfar, S.D. Katebi. User Interface Designfor E-Learning Software.In: The International Journal of Soft
Computing and Software Engineering [JSCSE], Vol. 3, No. 3„
January 2014.
2. J. Hou, J. Rashid, K. Lee, H. Kopka. Cognitive map or medium
materiality? Reading on paper and screen.In: Human Behavior,February 2017.
3. Z. Zhan, L. Zhang, H. Mei, P.S.W. Fong. Online Learners’
Reading Ability Detection Based on Eye-Tracking Sensors. In:Sensors, September 2016.
4. M. E. Dawson, A.M. Schell, D.L. Filion. The Electrodermal
System. In: The Handbook of Psychophysiology. Cambrige
University, 2007
5. A. Greco, A. Lanata, G. Valenza, E.P. Scilingo,
L. Citi.Electrodermal Activity Processing: a Convex Optimization
Approach. DOI10.1109/EMBC.2014.6944077, August 2014.