Using eye-tracking technology as an indirect instruction tool to improve text and picture processing and learning

This study used an eye-movement modelling example (EMME) in the school context to corroborate and extend recent findings about the educational potential of eye-tracking technology for supporting strategic processing and learning from an illustrated text. Sixty-four seventh graders were randomly assigned to the modelling and non-modelling conditions to investigate whether (1) those with the opportunity to observe a model’s eye movements while reading an illustrated text show greater integrative processing in their own reading and (2) they learn more deeply from text. Findings reveal that the students who observed the model’s visual behaviour showed greater integrative processing of text and picture. They made more transitions from one representation to the other and strategically spent longer re-inspecting the picture while rereading the text and vice versa. These students also outperformed those in the non-modelling condition for deeper learning as revealed in the transfer of knowledge task. Moreover, students with lower reading comprehension skills benefitted more from observing the model’s gaze replay when considering both the acquisition and transfer of knowledge. Introduction and rationale This study aimed to corroborate and extend current research on the potential of eye-tracking technology in the educational context to model one of the most common activities carried out to learn disciplinary knowledge, that is, reading an illustrated text. It is well-known that eye-tracking technology can be used as a research tool in learning contexts to track cognitive processing while carrying out various tasks (eg, Wang, Chen & Lin, 2014; Zheng & Cook, 2012). It provides unique information concerning perceptual and cognitive processes that underlie learning performances (van Gog & Scheiter, 2010). Recently, this technology has attracted interest in educational research on multimedia learning (Eitel, Scheitel, Schüler, Nyström & Holmqvist, 2013; Mason, Pluchino & Tornatora, 2013; Mason, Tornatora & Pluchino, 2013). What is less known is that eye-tracking technology can be used as an indirect instruction tool for sustaining the processes and outcomes of learning in educational contexts (Jarodzka et al, 2012; Mason, Pluchino & Tornatora, 2015; Seppänen & Gegenfurtner, 2012). In other words, eyetracking technology has been increasingly used as a tool to investigate the allocation of attention while executing of crucial tasks and its relation with learning performance. However, eyetracking technology has not been used yet as an educational technology in the classroom. How British Journal of Educational Technology (2015) doi:10.1111/bjet.12271 © 2015 British Educational Research Association can this technology become educational? Nowadays, there are non-intrusive eye-trackers that not only record eye-movements while performing a learning activity, but also make gaze replays available in the form of videos. These videos can easily be used in the service of learning. In this regard, eye-movement modelling examples (EMMEs) is a recent technology-supported tool in which videos of the gaze replays of an expert model a learner’s behaviour. Specifically, the position of a skilful expert’s eye-movements while performing an activity is shown to less skilful learners to help them carry out the same activity (van Gog, Jarodzka, Scheiter, Gerjets & Paas, 2009). Therefore, eye-tracking technology becomes an educational technology when is used to teach—indirectly—how to perform a learning activity well. Theoretically rooted in observational learning (Bandura, 1977), EMMEs are a special case of video-based modelling that has been increasingly used in education to model various task and activities (eg, Groenendijk, Janssen, Rijlaarsdam & van den Bergh, 2013). In the present study, we focused on modelling the strategic reading of an illustrated science text for two reasons: (1) it is crucial for successful learning in the school context given that almost all textbooks include text and illustrations and (2) students usually spend more time reading the verbal parts than they do inspecting the graphical parts of learning materials (Cromley, Snyder-Hogan & Luciw-Dubas, 2010a; Hannus & Hyönä, 1999; Schmidt-Weigand, Kohnert & Glowalla, 2010) and are often under the illusion that they comprehend illustrations (Schroeder et al, 2011). Practitioner Notes What is already known about this topic • Successful learning from an illustrated text requires the integration of verbal and graphical information. • Often students do not pay much attention to pictures as they believe they comprehend them very easily. • Eye-tracking technology can be used as a research tool to track cognitive processing while reading. What this paper adds • Students who observed a video of a model’s eye-movements while reading an illustrated text showed a more integrative processing of text and picture during their own reading of an illustrated text on a different topic. • Students who observed the model’s visual behaviour were also superior in the task requiring the transfer of new knowledge. • Students with low reading comprehension skills benefitted more from observing the video with the model’s gaze replay than students with higher reading comprehension skills. Implications for practice and/or policy • Eye-tracking technology has the potential to be an educational technology. The study suggests that it can be used to model one of the most common learning activities in the school context, that is reading an illustrated informational text. • The potential benefit of an eye-movement modelling example seems to extend to deeper learning from text. • Less skilled readers in particular can benefit more from observing a model’s visual behaviour while strategically reading an illustrated informational text. 2 British Journal of Educational Technology © 2015 British Educational Research Association When learning complex concepts, multimedia materials are potentially beneficial for the different functions that they offer (Ainsworth, 2006). The current study deals with the most common multimedia materials used in schools—texts accompanied by instructional pictures. Examining how students process and learn from illustrated texts is important for understanding how this crucial learning activity can be supported to make it more effective. Theoretical accounts of learning from text and pictures (Mayer, 2009, 2014; Schnotz, 2014; Schnotz, Baadte, Johnson & Mengelkamp, 2012) agree that integrating verbal and graphical information is crucial for learning better from an illustrated text than from a text alone. Eyetracking studies have investigated how the integration of text and pictures can be fostered by presenting signals (Jamet, 2014). For example, the most important terminology is written in the same colour within the text and the picture to increase its salience (Ozcelik, Arslan-Ari & Cagiltay, 2010; Ozcelik, Karakus, Kursun & Cagiltay, 2009). In this study, we sought to sustain the integration of verbal and graphical information in a different way, that is, through the use of eye-movement modelling in the school context. To our knowledge, only one study has been carried out using this technology-supported tool for modelling lower secondary school students’ integrative behaviour while reading an illustrated text to learn conceptual knowledge from it (Mason et al, 2015). In the present study, we sought to corroborate and extend the positive findings of this previous investigation. Corroborating means providing new evidence for the educational potential of eye-tracking technology in sustaining integrative processing during the more strategic second-pass, that is, rereading, in the service of learning. Extending the previous findings means examining its educational potential in relation to the crucial individual characteristic of reading comprehension skill which, by definition, is related to learning from text (Cromley, Snyder-Hogan & Luciw-Dubas, 2010b). Specifically, three research questions (RQs) guided our study. RQ1: Does an EMME of integrative processing of text and picture in reading an illustrated text help students to integrate more verbal and graphical information during their own reading of an illustrated text? RQ2: Does the EMME of integrative processing of text and picture also help students to learn more deeply from the illustrated text? RQ3: Do reading comprehension skills moderate the effect of the EMME on learning? Based on previous research (Mason et al, 2015), we formulated the following hypotheses. H1: Students with the opportunity to observe a model’s gaze replay would show stronger integrative processing as indicated by the frequency of gaze shifts from the verbal to the graphical representation and vice versa, and the duration of their re-inspection of the graphical information while rereading the verbal segments and vice versa, during the delayed and more purposeful second-pass reading. H2: Students who observed the model’s gaze replay would also perform better in the transfer of knowledge. More than surface learning, this measure of deeper learning should require the integration of verbal and graphical information. H3: Reading comprehension skill would moderate the benefit of the EMME, helping low-comprehenders to benefit more from the modelling of strategic reading of an illustrated text, according to an aptitudetreatment interaction. This refers to the fact that some instructional strategies (treatments; in this case the EMME) are more or less effective for particular individuals depending on their specific abilities (aptitudes; in this case reading comprehension) (Park & Lee, 2003; Snow & Swanson, 1992).