2017 14th IAPR International Conference on Document Analysis and Recognition

Cognitive State Measurement on Learning Materials
by Utilizing Eye Tracker and Thermal Camera
Shoya Ishimaru∗† , Soumy Jacob† , Apurba Roy† , Syed Saqib Bukhari∗† ,Carina Heisel† , Nicolas Großmann† , Michael Thees† , Jochen Kuhn† and Andreas Dengel∗†∗ German Research Center for Artificial Intelligence (DFKI)

Kaiserslautern, Germany
E-mail: {firstname.lastname}@dfki.de† University of Kaiserslautern
Kaiserslautern, Germany

Abstract—We demonstrate how information derived from pervasive sensors can quantify cognitive states of learners while they
are reading a textbook. Eye tracking is one of the most effective
approaches to measuring reading behavior. For example, highfixation duration represents a reader’s attention on a document.
However, it is still a challenging task to predict the reason for
the attention (i.e., is it because of his/her interest or troubleof understanding?). In this paper, we utilize additional sensing
modality to solve the problem. On the dataset of 12 high school
students’ reading behaviors, we have found that the changing
of pupil diameter and nose temperature are highly correlatedwith their cognitive states including their interests and efforts
for reading/solving tasks on learning materials in Physics.

I. I NTRODUCTION

Fig. 1. The key idea of our work. Combining eye tracking and thermography
to recognize cognitive states while learners are reading a document.

Documents have been playing an important role in learningand education. Still today, various school subjects are alignedwith contents which are captured in textbooks. Althoughcuriosity is an important factor for learning, every student
has a different way of learning based on individual speed
and preferences, textbooks have traditionally been found to bestatic and consistently dull for a variety of learners. Therefore,
students sometimes avert their eyes from reading a textbook
because it is boring. We believe that one of the solutions to thisproblem is to apply Human-Document Interaction in textbook
reading, i.e., to develop a digital textbook which can make thematerials for learning and instruction dynamic and anticipating
on display. Especially in Physics, it is highly efficient to show
phenomenon, experiments, representations and 3D models as
dynamic contents using multi medias.
The idea of making texts dynamic has been originallyproposed by Biedert et al. as Text2.0 [1]. They have created aframework to construct gaze-responsive realtime interactions
to enhance the reading experience (e.g., displaying images,
translations, footnote, and bookmarks). But even if the various
interactions are actuated by each reader’s reading behavior, theaugmentations are same among all readers. The motivationof our study is to make texts not only dynamic but also
anticipating. We measure the cognitive states of learners
including their interests, workloads, and comprehensions, thenaugment the text by providing individualized information toenhance their learning abilities.
2379-2140/17 $31.00 © 2017 IEEE
DOI 10.1109/ICDAR.2017.378

To develop such an anticipating textbook, it is necessary toinvestigate what kind of features can be used to recognize thelearners’ cognitive states. One of the most effective approach
to investigate reading activity is to utilize eye tracking. Thereis extensive research on eye movements during reading in
psychology and cognitive science [2]. High fixation duration,for example, represents a reader’s attention on a document [3].
However, it is still a challenging task to recognize from eye
movements alone, the reason for his/her attention, interest
or difficulty in comprehension. In this paper, as shown in
Figure 1, we utilize additional sensing modality to solve theproblem. Based on the literature review (refer to Section II),
we follow the idea of measuring the change of the nose
temperature. We prepared a learning material about “Basic
Phenomena in Acoustics and Pendulum” including teachingtexts and related exercises, and asked high school students toread. The specific contributions of our work are two-folds:
Demonstrating that the combination of commercial infrared thermal camera and facial landmark detection isaccurate enough to measure students’ nose temperatures.
• Investigating effective features measured by an eye
tracker and an infrared thermal camera which are related
to cognitive states collected as self-assessments.
•

32

Fig. 3. Examples of changing of nose temperature of two participants (top:low workload, bottom: high workload) during they are reading the textbook(red) and solving the tasks (orange). The x-axis represents timestamps (sec.).

Fig. 2. Fixations on a display during solving a task. The colors represent theorder (from red to blue) and the durations are visualized as radiuses.

II. R ELATED WORK

reading [13]. Bulling et al. have explored tracking eye movements in natural settings using Electrooculography glasses.
They looked into reading, as well as other everyday activities
and cognitive processes [14]. Features from eye movements
can predict the volume of reading (counting the number ofread words) [15], [16], a level of English skill [17], anddifficult word for a reader [18]. As these authors employ
general features from the eye (e.g. fixation duration, saccade
length and velocity), their approach can be used on a Physics
textbook as well.
Our study is especially close to research by Mozaffari etal. [19] and Ishimaru et al. [20]. They have recorded eye
movements of high school students during they are reading andsolving Physics tasks to investigate specific gaze patterns forpredicting expertise of students. Compared to their research,
the position of our study is to investigate not only the performances of students but cognitive states including interest andworkload by sensing devices.

A. Cognitive state analyses in Physics education
Competent handling of multiple representations is supposed to be significant for learning and problem- solving inphysics [4], [5]. A psychological model for understanding
the cognitive processes while working with multiple representations is offered by the Cognitive Theory of Multimedia Learning (CTML; [6]) and the Cognitive Load Theory(CLT; [7]). Referred to as CTML, the generation of a mental
model of the learning content requires an active part in information processing. The presentation format of the learningmaterial is essential and can be structured into text/picture
or classified according to dynamics and interactivity [8]. Students’ learning is improved by presenting text/equations andpictures/graphs/videos instead of learning with text/equations
alone. While using the pictorial and verbal/auditory channels
simultaneously, sensory and representational differentiations
are connected. As a result, cognitive load is reduced. So,
greater capacity of working memory is available for forming
mental representation models according to CTML and, therefore, learnability is increased.
Besides enhancing cognitive variables, our approach involves that students read textbooks actively through personalized feedback to learn in their way. So, they experience
autonomy, which is said to foster motivation in general [9],[10] as well as curiosity, as special features of motivation, inparticular [11], [12].

C. Medical and physiological sensing
The obvious approach to measure cognitive states is tomeasure our brain. Electroencephalography (EEG) [21] andnear-infrared spectroscopy (NIRS) [22] can be candidates, andperform well to recognize activities and cognitive states. But
they are too bulky to be worn regularly by considering theapplication scenario that real school students use the system.
On the other hand, autonomic nervous system (ANS) including the sympathetic nervous system and the parasympathetic
nervous system is a measurable component of emotional
response [23]. The nose temperature drops when a person feels
high workload [24] and high engagement on reading [25]. The
diameter of the pupil also reflects the workload. The relation
has been investigated in the tasks of memory [26] and driving
a car [27]. Compared to sensing brain activation, these signals
have an advantage that they can be measured remotely withoutwearing devices.

B. Eye tracking
Most of the studies mentioned above obtain students’ insights from only answering sheets afterward and do not careabout their learning process while reading texts and solving
tasks. One of the interesting approaches to get realtime insights
into cognitive behavior is to employ gaze data. Our vision
is to enhance physics teaching and learning and develop
possibilities for utilizing eye tracking. Rayner has investigated
the relationship and has provided a good summary related to33

TABLE I
T HIRTEEN SURVEYS

Fig. 4. An experimental setup using an eye tracker and a thermal camera.

id
s1
s2

type
interest
interest

scope
macro
macro

s3
s4

interest
interest

micro
micro

s5

interest

micro

s6
s7
s8
s9
s10

confidence
confidence
confidence
workload
workload

macro
micro
micro
micro
micro

s11
s12

workload
workload

micro
micro

s13

expertisemacro

surveyI enjoy solving physics problems.
I am concerned about homework with topics
dealing with physics.
I like the content of the textbook.
I am interested in learning more about the subjectof the textbook as well as lectures and homework.
I would like to know more on the topic of
textbook in school.
I am good at physics more than other subjects.
The textbook text was easy to understand.
I knew what I had to answer during solving the tasks.
I had to make an effort to solve the questions.
It was difficult to find the right information to solve
the questions in the text.
I needed more assistance while reading the textbook.
The textbook made me curious to know more about
vibration and acoustics.
My physics record is about....

shown in Figure 3. From an initial analysis, we have found thatgenerally, the temperature increases when the students readthe textbook and decreases when they start solving exercises.
From this data, the slope and the standard deviation of theparticipant’s temperature during the process of solving eachquestion are calculated. Finding out the slope and standard
deviation serves to measure the ascend/descend and the fluctuations in temperature.

III. A PPROACH
Based on the literature review, we select the combination ofeye tracking and thermal image analysis to measure cognitive
states because they can be sense without bothering readersand do not interfere with each other. This section describes
the pre-processing and feature calculations of the sensing.
A. Eye tracking
We utilize a remote eye tracker which can attached to adisplay to track eye movements. Eye gaze data is composedof two metrics - fixations and saccades. A fixation occurs
when the gaze falls on something of interest to the screen
area and usually lasts for about 100 - 150 ms. The rapid
movement of the eye between fixations is called a saccade. As
preprocessing, we filter raw eye movements to fixations andsaccades on the basis of the approach proposed by Buscher etal [28]. Figure 2 shows one of the examples of the filtering.
The average of left and right pupil diameters at any time
instant is used as the pupil diameter feature for this work. The
duration of each fixation during each question is aggregatedto get the fixation duration feature. The length of a saccadeis derived from the known values of fixation duration of theeye at a particular 2-dimensional coordinate on the screen, at
a given timestamp. Similar to the fixation duration feature,
the summation of the saccade length corresponding to eachparticipant for each question is used to obtain the feature value.
The mean and standard deviation of fixation durations andsaccade lengths are calculated as features.

IV. E XPERIMENT
We have recorded students’ reading behaviors and investigated effective features to recognize cognitive states. In thefollowing, we present the experimental setup, the analysis
results, and findings from this experiment.
A. Experimental setup
Figure 4 shows an overview of the experimental setup. The
SMI 60 Hz remote eye tracker was set up alongside a normal
computer desktop to record the eye gaze data and FLIR One
for iOS was set to capture the thermal energy from the face.
The eye tracker uses a reflection of infrared light to measureeye gaze. We made sure that there is no significant affectin thermal images before starting the experiment. We asked
fourteen sixth-grade students (11 or 12 years old) to participate
in the experiment. They read a Physics textbook on a screen
and solved eight exercises related to the content. As shown in
Figure 4, the content textbook was displayed on the left pageon the screen, and exercises were displayed on the right page.
Participants are allowed to use a calculator while solving theexercises. Eye movements on the calculator were excluded in
the analysis. Note that 7 participants read the text first andother 7 participants read questions first. But we treat them as
the same condition because there are no significant differencesin their performances calculated by the score of the exercises.
To collect ground truth of cognitive states, we asked
participants to answer surveys on a paper form after therecording. We prepared the surveys as shown in Table I fromthe viewpoint of Physics education research. They can be
categorized with two indexes: the type and the scope. We asked

B. Nose temperature tracking
We utilize FLIR One for iOS, a commercial thermal camera
which can be attached to a smartphone or a mobile tablet
to measure face temperatures. We develop the sensor logging
application of the device by ourselves and record the changing
of temperatures as a video. Positions of the face and the noseon each frame are detected by utilizing the method proposedby Baltrusaitis et al. [29]The temperature data consists of the nose temperature ofeach participant at a given time during the experiment as

34

TABLE II
P EARSON C ORRELATION AND p- VALUES FEATURES AND THIRTEEN SURVEYS .
featurenose slope reading
nose slope solving
nose std reading
nose std solving
pupil mean reading
pupil mean solving
pupil std reading
pupil std solving
fixation mean reading
fixation mean solving
fixation std reading
fixation std solving
saccade mean reading
saccade mean solving
saccade std reading
saccade std solving

s1-0.0 (0.97)-0.3 (0.39)
0.0 (0.92)
0.6 (0.04)
0.6 (0.03)
0.7 (0.01)
0.4 (0.16)
0.2 (0.57)-0.4 (0.25)-0.3 (0.29)-0.3 (0.30)-0.4 (0.26)
0.1 (0.81)
0.3 (0.28)
0.0 (0.96)
0.3 (0.35)

s2
0.3 (0.37)
0.3 (0.40)
0.3 (0.38)-0.4 (0.22)-0.6 (0.03)-0.6 (0.06)-0.4 (0.23)-0.4 (0.21)
0.3 (0.36)
0.0 (0.91)
0.2 (0.63)-0.1 (0.84)-0.6 (0.05)-0.3 (0.41)-0.5 (0.10)-0.2 (0.61)

interests3
0.4 (0.23)-0.4 (0.25)
0.4 (0.18)
0.0 (0.90)
0.7 (0.01)
0.7 (0.01)
0.8 (0.00)
0.6 (0.03)-0.4 (0.18)-0.4 (0.16)-0.3 (0.40)-0.1 (0.83)-0.1 (0.82)
0.4 (0.22)-0.1 (0.80)
0.4 (0.18)

s4
0.0 (0.88)-0.0 (0.96)
0.1 (0.73)-0.1 (0.76)
0.1 (0.72)
0.1 (0.80)
0.4 (0.22)
0.6 (0.06)-0.1 (0.87)
0.0 (0.99)-0.1 (0.86)-0.1 (0.83)
0.2 (0.50)-0.3 (0.41)
0.3 (0.28)-0.0 (0.96)

s5
0.0 (0.89)
0.2 (0.53)
0.1 (0.84)-0.3 (0.42)
0.5 (0.14)
0.5 (0.12)
0.5 (0.14)
0.4 (0.25)-0.1 (0.80)-0.2 (0.51)-0.0 (0.89)-0.1 (0.68)-0.1 (0.75)-0.3 (0.29)-0.1 (0.68)-0.2 (0.49)

s6
0.2 (0.60)
0.6 (0.04)
0.0 (0.88)-0.2 (0.58)
0.1 (0.80)
0.1 (0.76)-0.1 (0.67)-0.3 (0.40)
0.0 (0.97)-0.2 (0.44)-0.0 (0.97)-0.2 (0.44)-0.0 (0.93)-0.3 (0.42)-0.1 (0.87)-0.2 (0.50)

confidence
s7-0.2 (0.49)
0.1 (0.74)-0.2 (0.55)-0.4 (0.19)
0.1 (0.88)
0.1 (0.77)-0.0 (0.92)-0.0 (1.00)
0.3 (0.42)
0.5 (0.14)
0.2 (0.51)
0.5 (0.11)-0.6 (0.02)-0.5 (0.10)-0.7 (0.01)-0.5 (0.11)s8
0.1 (0.75)
0.4 (0.22)
0.3 (0.40)-0.6 (0.03)-0.7 (0.01)-0.7 (0.02)-0.4 (0.20)-0.2 (0.47)
0.5 (0.08)
0.4 (0.19)
0.4 (0.24)
0.1 (0.70)-0.5 (0.14)-0.6 (0.03)-0.3 (0.29)-0.5 (0.12)s9
0.1 (0.73)-0.7 (0.01)
0.2 (0.51)
0.1 (0.73)
0.3 (0.39)
0.3 (0.27)
0.5 (0.07)
0.5 (0.13)-0.1 (0.85)
0.1 (0.81)-0.1 (0.73)
0.2 (0.52)-0.5 (0.12)
0.3 (0.43)-0.4 (0.26)
0.3 (0.42)

workloads10s11-0.5 (0.09)-0.2 (0.50)-0.8 (0.00)-0.2 (0.48)-0.4 (0.15)-0.1 (0.76)
0.3 (0.29)
0.2 (0.57)
0.5 (0.10)
0.5 (0.08)
0.5 (0.12)
0.5 (0.10)
0.5 (0.09)
0.4 (0.25)
0.6 (0.03)
0.3 (0.40)-0.1 (0.83)-0.2 (0.58)
0.3 (0.32)-0.4 (0.24)-0.1 (0.87)
0.1 (0.76)
0.4 (0.26)-0.2 (0.53)-0.0 (0.95)
0.6 (0.05)
0.1 (0.74)
0.3 (0.41)-0.0 (0.94)
0.5 (0.10)
0.1 (0.85)
0.1 (0.64)

s12
0.2 (0.53)
0.3 (0.41)-0.1 (0.73)
0.1 (0.70)-0.0 (1.00)-0.2 (0.63)
0.0 (0.99)
0.0 (0.91)-0.5 (0.14)-0.3 (0.33)-0.3 (0.31)
0.1 (0.84)
0.7 (0.02)
0.2 (0.50)
0.7 (0.01)
0.4 (0.24)

expertises13-0.2 (0.61)-0.4 (0.16)
0.1 (0.85)
0.1 (0.71)-0.2 (0.50)-0.1 (0.71)-0.1 (0.69)-0.2 (0.57)
0.3 (0.37)
0.2 (0.59)
0.2 (0.63)-0.1 (0.75)-0.4 (0.23)
0.2 (0.60)-0.4 (0.24)-0.1 (0.87)

Fig. 5. Pearson correlation between surveys and slope of nose temperature.

Fig. 6. Pearson correlation between surveys and mean of pupil diameter.

three types of subjective cognitive states (interest, confidence,
workload) plus one objective measurement (expertise). Someof the surveys are general questions about Physics learning(macro) and the others are specific questions about the content(micro). The survey brought to light the interest these students
had in learning and researching about physics. The ratings
ranged from 1 to 6, 6 being “I agree completely and wholly”
and 1 being “I do not agree with it at all”. Note that there are
two differences about the survey between the original form
used in the experiment and reported in this paper. 1) The orderof the surveys in the form during the experiment was s7, s3,
s8, s6, s12, s11, s1, s10, s4, s2, s9, s5, and s13. In this paper,
they are sorted by their semantic. 2) The surveys were written
in German during the experiment. They are translated into
English in the table for readers of this paper.
We reluctantly exclude two of 14 participants’ data as
outliers. Their reading and solving time were too fast thanother participants, they seemed to select the answers randomly,
and their score of exercises were zero. They could not be
attentive enough or understand the purpose of the experiment.

From these values, we have found three insights. First, surveys
related to workload including s10 “It was difficult to find theright information to solve the questions in the textbook.” ands9 “I had to make an effort to solve the questions.” can bemeasured by a decrease of the nose temperature during solving
exercises (p = 0.001 and p = 0.012). Second, increase of pupil
diameter represents a student’s interest including s3: “I likethe content of the textbook.” and s1 “I enjoy solving physics
problems.” (p = 0.008 and p = 0.030 during reading; p = 0.006
and 0.013 during solving). Third, students who read a textbook
and exercises with small saccades felt high confidence intheir understandings reflected in s7 “The textbook was easy
to understand” and s8 “I knew what I had to answer duringsolving the tasks” (p = 0.025 and p = 0.035). Details of therelation between surveys and effective features, the slope ofnose temperature and mean of pupil diameter are visualized
in Figure 5 and 6 for clear visualizations.
C. Discussion
The temporal resolution of sensing is a remaining issue.
Although the change in the nose temperature is an effective
feature to understand a student’s effort, it requires a long time
to be observed (see Figure 3). In the application scenario, it canbe used for the measurement on each learning unit or page.
But it seems difficult to apply our measurements on small

B. Results
Table II represents the Pearson correlation and p-values (inbrackets) between the features and surveys. High correlations
with p-values less than 0.05 are highlighted as bold fonts.

35

parts such as each paragraph, image, or sentence. We need toinvestigate how much the time resolution can be minimized.

[11] S. Von Stumm, B. Hell, and T. Chamorro-Premuzic, “The hungry
mind intellectual curiosity is the third pillar of academic performance,”
Perspectives on Psychological Science, vol. 6, no. 6, pp. 574–588, 2011.
[12] F. F. Schmitt and R. Lahroodi, “The epistemic value of curiosity,”
Educational Theory, vol. 58, no. 2, pp. 125–148, 2008.
[13] K. Rayner, “Eye movements in reading and information processing: 20
years of research.” Psychological bulletin, vol. 124, no. 3, p. 372, 1998.
[14] A. Bulling and D. Roggen, “Recognition of visual memory recall
processes using eye movement analysis,” in Proceedings of the 13th
international conference on Ubiquitous computing. ACM, 2011, pp.
455–464.
[15] K. Kunze, K. Masai, M. Inami, Ö. Sacakli, M. Liwicki, A. Dengel,
S. Ishimaru, and K. Kise, “Quantifying reading habits: counting how
many words you read,” in Proceedings of the 2015 ACM International
Joint Conference on Pervasive and Ubiquitous Computing. ACM, 2015,
pp. 87–96.
[16] S. Ishimaru, K. Kunze, K. Kise, and A. Dengel, “The wordometer 2.0:
estimating the number of words you read in real life using commercial
eog glasses,” in Proceedings of the 2016 ACM International Joint
Conference on Pervasive and Ubiquitous Computing: Adjunct. ACM,
2016, pp. 293–296.
[17] K. Yoshimura, K. Kunze, and K. Kise, “The eye as the window of thelanguage ability: Estimation of english skills by analyzing eye movement
while reading documents,” in Proceedings of the 13th International
Conference on Document Analysis and Recognition, 2015, pp. 251–255.
[18] A. Okoso, T. Toyama, K. Kunze, J. Folz, M. Liwicki, and K. Kise,“Towards extraction of subjective reading incomprehension: Analysis ofeye gaze features,” in Proceedings of the 33rd Annual ACM Conference
Extended Abstracts on Human Factors in Computing Systems. ACM,
2015, pp. 1325–1330.
[19] S. Mozaffari, S. Bukhari, A. Dengel, P. Klein, and J. Kuhn, “A study
on representational competence in physics using mobile remote eye
tracking systems, proceedings,” in Proceedings of the 18th International
Conference on Human-Computer Interaction with Mobile Devices andServices, 2016, accepted.
[20] S. Ishimaru, S. S. Bukhari, C. Heisel, J. Kuhn, and A. Dengel, “Towards
an intelligent textbook: eye gaze based attention extraction on materials
for learning and instruction in physics,” in Proceedings of the 2016 ACM
International Joint Conference on Pervasive and Ubiquitous Computing:
Adjunct. ACM, 2016, pp. 1041–1045.
[21] K. Kunze, Y. Shiga, S. Ishimaru, and K. Kise, “Reading activity
recognition using an off-the-shelf eeg – detecting reading activities anddistinguishing genres of documents,” in Proceedings of the 2013 12th
International Conference on Document Analysis and Recognition, ser.
ICDAR ’13. IEEE Computer Society, 2013, pp. 96–100.
[22] S. Ishimaru, K. Kunze, K. Kise, and M. Inami, “Position paper: Brainteasers - toward wearable computing that engages our mind,” in Proceedings of the 2014 ACM International Joint Conference on Pervasive andUbiquitous Computing: Adjunct Publication, ser. UbiComp ’14 Adjunct.
ACM, 2014, pp. 1405–1408.
[23] R. M. Sapolsky, Why zebras don’t get ulcers. WH Freeman New York,
1994.
[24] P. Karthikeyan, M. Murugappan, and S. Yaacob, “Descriptive analysis
of skin temperature variability of sympathetic nervous system activity
in stress,” Journal of Physical Therapy Science, vol. 24, no. 12, pp.
1341–1344, 2012.
[25] K. Kunze, S. Sanchez, T. Dingler, O. Augereau, K. Kise, M. Inami,
and T. Tsutomu, “The augmented narrative: toward estimating reader
engagement,” in Proceedings of the 6th augmented human international
conference. ACM, 2015, pp. 163–164.
[26] D. Kahneman and J. Beatty, “Pupil diameter and load on memory,”
Science, vol. 154, no. 3756, pp. 1583–1585, 1966.
[27] O. Palinko, A. L. Kun, A. Shyrokov, and P. Heeman, “Estimating
cognitive load using remote eye tracking in a driving simulator,” inProceedings of the 2010 symposium on eye-tracking research & applications. ACM, 2010, pp. 141–144.
[28] G. Buscher, A. Dengel, and L. van Elst, “Eye movements as implicit
relevance feedback,” in CHI’08 extended abstracts on Human factors incomputing systems. ACM, 2008, pp. 2991–2996.
[29] T. Baltrusaitis, P. Robinson, and L.-P. Morency, “Constrained local
neural fields for robust facial landmark detection in the wild,” inProceedings of the IEEE International Conference on Computer Vision
Workshops, 2013, pp. 354–361.

V. C ONCLUSION AND F UTURE W ORKThis paper demonstrated the cognitive state analysis byusing an eye tracker and an infrared thermal camera. We havedeveloped an application to retrieve the change in the nose
temperature from a commercial infrared thermal camera (FLIR
ONE). We asked 12 high school students to read/solve learningmaterials in Physics and investigated the relation between
sensor signals and surveys about their cognitive states. Thechanging of the pupil diameter was highly correlated withinterest. Although the temporal resolution was not enoughhigh for a realtime application scenario, the changing of thenose temperature represented their efforts for reading/solving
learning materials.
This work investigated “when” additional information
should be displayed on an anticipating textbook based on
the measurement of cognitive states. We plan to implement
dynamic changing on textbooks, and investigate two researchquestions. What kind of additional information can improvestudents’ learning abilities? What is the best way to display
additional information overcoming the split attention effect?
ACKNOWLEDGMENT
This work is funded by the Federal Ministry of Education and Research (BMBF) in the framework of the joint
initiative ”Qualitätsoffensive Lehrerbildung” of the Federal
and the Federal States of Germany for the project ”U.EDU:
Unified Education” (support code: 01JA1616). The authors are
responsible for the content of this contribution.
R EFERENCES[1] R. Biedert, G. Buscher, S. Schwarz, M. Möller, A. Dengel, and T. Lottermann, “The text 2.0 framework: writing web-based gaze-controlled
realtime applications quickly and easily,” in Proceedings of the 2010
workshop on Eye gaze in intelligent human machine interaction. ACM,
2010, pp. 114–117.
[2] K. Rayner, “Eye movements in reading and information processing.”
Psychological bulletin, vol. 85, no. 3, p. 618, 1978.
[3] K. Rayner, K. H. Chace, T. J. Slattery, and J. Ashby, “Eye movements
as reflections of comprehension processes in reading,” Scientific Studies
of Reading, vol. 10, no. 3, pp. 241–255, 2006.
[4] J. Dolin, “Science education standards and science assessment in denmark,” Making it comparable: Standards in science education, pp. 71–
82, 2007.
[5] S. Ainsworth, “Deft: A conceptual framework for considering learningwith multiple representations,” Learning and instruction, vol. 16, no. 3,
pp. 183–198, 2006.
[6] W. Schnotz, “The cambridge handbook of multimedia learning: An
integrated model of text and picture comprehension,” 2005.
[7] P. Chandler and J. Sweller, “Cognitive load theory and the format of
instruction,” Cognition and instruction, vol. 8, no. 4, pp. 293–332, 1991.
[8] R. Girwidz, T. Rubitzko, S. Schaal, and F. Bogner, “Theoretical concepts for using multimedia in science education,” Science Education
International, vol. 17, no. 2, pp. 77–93, 2006.
[9] R. M. Ryan and E. L. Deci, “Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being,”
American psychologist, vol. 55, no. 1, p. 68, 2000.
[10] ——, “Intrinsic and extrinsic motivations: Classic definitions and new
directions,” Contemporary educational psychology, vol. 25, no. 1, pp.
54–67, 2000.

36