Does Spatial Verbal Information Interfere with Picture Processing in Working Memory? The Role of the Visuo-spatial Sketchpad in Multimedia Learning

Does Spatial Verbal Information Interfere with Picture Processing in Working Memory? The Role of the Visuo-spatial Sketchpad in Multimedia Learning Anne Schuler (a.schueler@iwm-kmrc.de) Knowledge Media Research Center, Konrad-Adenauer-Strasse 40, 72072 Tuebingen, Germany Katharina Scheiter (k.scheiter@iwm-kmrc.de) Knowledge Media Research Center, Konrad-Adenauer-Strasse 40, 72072 Tuebingen, Germany Peter Gerjets (p.gerjets@iwm-kmrc.de) Knowledge Media Research Center, Konrad-Adenauer-Strasse 40, 72072 Tuebingen, Germany cognitive resources so that an overload in one or both systems can be avoided. However, since Baddeley’s first comprehensive descriptions of his model there have been numerous new findings concerning the functioning of working memory that have been considered in newer versions of the Baddeley model, but have not yet been incorporated into the CTML. In particular, the structure of the VSSP has been further specified. According to our view, these specifications may play an important role in multimedia learning. Thus, the aim of this paper is to have a closer look at the VSSP and its implications for learning with multimedia. Abstract The reported study examined whether the processing of spatial verbal information interferes in the visuo-spatial sketchpad with the execution of eye movements, associated with viewing pictures and reading. Seventy-four students were randomly assigned to six groups, resulting from a 2×2×2 mixed design, with spatial secondary task (with vs. without), text contents (visual vs. spatial), and text modality (spoken vs. written) as independent variables. Consistent with our assumptions, learners with text containing spatial contents showed worse recall performance than those with text containing visual contents. Furthermore, written presentation of text containing spatial contents loaded the visuo-spatial sketchpad to a higher extent than spoken presentation. Implications of these results for learning with multimedia are discussed. A Closer Look at the Visuo-spatial Sketchpad Keywords: multimedia; working memory; modality effect; spatial verbal information; secondary task Introduction In the last two decades, a lot of research has been conducted on how people learn from multimedia, that is, from the presentation of texts together with pictures (Mayer, 2009). The Cognitive Theory of Multimedia Learning (CTML; Mayer, 2009) is one of the most important theories concerning multimedia learning. One of its theoretical foundations is an older version of Baddeley’s working memory model (1992). According to this model, working memory consists of three systems: The phonological loop (PL), where all verbal information is processed, the visuo- spatial sketchpad (VSSP), where visual and spatial information is processed, and the central executive, which governs the functioning of the phonological loop and the VSSP. Accordingly, the CTML assumes that texts are processed in the phonological loop, whereas pictures are processed in the VSSP. The working memory systems are limited in the amount of information that can be processed in parallel. Accordingly, processes accomplished within the same system can interfere with each other and hinder learning. Therefore, text-picture presentations should be designed in a way that a learner can make optimal use of the According to Logie (1995), the VSSP can be divided into a visual and a spatial part. Whereas the visual part deals with information like an object’s color or form, the spatial part handles information like spatial sequences or spatial configurations (e.g., Darling, Della Sala, & Logie, 2007; Della Sala et al., 1999). Whereas Logie and colleagues focused on pictorial stimuli, other researchers have addressed the question whether the VSSP may also be involved in the processing of text. This research suggests that if text contains information about spatial and/or visual configurations, it will not be processed only in the PL but also in the respective part of the VSSP, whereas if it contains more abstract information, it will be processed in the PL alone (De Beni et al., 2005; Deyzac, Logie, & Denis, 2006). Another line of research on the spatial VSSP has also shown that this structure is not responsible only for the processing of spatial information but also for the control of movements, for example arm or eye movements (e.g., Postle et al., 2006). Although from a theoretical perspective the VSSP should play a crucial role in multimedia learning, its involvement has not often been considered empirically in multimedia learning. One method to measure the involvement of the spatial VSSP in task performance is the secondary task paradigm. In this paradigm, two tasks are combined, a primary and a secondary task. The primary task is the main