Watching Gestures during Learning about Movements with Dynamic Visualization Activates the Human Mirror Neuron System: A fNIRS Study
2013-01-01

Watching Gestures during Learning about Movements with Dynamic Visualization Activates the Human Mirror Neuron System: A fNIRS Study Birgit Imhof (b.imhof@iwm-kmrc.de) a Ann-Christine Ehlis (ann-christine.ehlis@med.uni-tuebingen.de) b Florian B. Hausinger (florian.haeussinger@med.uni-tuebingen.de) b Peter Gerjets (p.gerjets@iwm-kmrc.de) a a b Knowledge Media Research Center, Schleichstrasse 6, 72076 Tuebingen, Germany Department of Psychiatry & Psychotherapy, University Hospital Tuebingen, Calwerstr. 14, 72076 Tuebingen, Germany Abstract This study investigates whether viewing human gestures facilitates learning about non-human biological movements and whether correspondence between gesture and to-be- learned movement is superior to non-correspondence. Functional near-infrared-spectroscopy was used to address whether gestures activate the human mirror-neuron-system (hMNS) and whether this activation mediates the facilitation of learning. During learning participants viewed triples of visualizations (animation – gesture video – animation). Results showed that for low-visuospatial-ability learners corresponding gestures led to higher cortical activation in the inferior-frontal cortex (part of the hMNS) and better learning outcomes, whereas for high-visuospatial-ability learners the type of gesture had no influence. Furthermore, results showed that – if presented with non-corresponding gestures – only low-visuospatial-ability learners who activated their inferior- parietal cortex (also part of the hMNS), improve their learning. Thus, activating the hMNS facilitates learning about movements and stimulating the hMNS via gestures seems to be an adequate instructional strategy to enhance learning with dynamic visualizations for low-visuospatial-ability learners. Keywords: Learning about movements; visualizations; human mirror-neuron-system; functional near-infrared-spectroscopy. dynamic gestures; Learning about Movements with Dynamic Visualizations Many contents in the Natural Sciences as well as in other domains, such as different sport disciplines or scene perception, comprise the understanding of changes in space over time. Dynamic visualizations can easily depict such changes and they may be particularly suited for instructional purposes if these changes do not occur in a discrete or linear way, but rather involve more complex continuous aspects (e.g., acceleration). However, they were not always superior to static visualizations to convey dynamic information (e.g., Imhof et al., 2012). Thus, it is crucial to understand when and for whom dynamic visualizations are beneficial to use them effectively and to exploit their potential for learning. Until now, research on the instructional use of dynamic visualizations has yielded rather heterogeneous results: Not only design factors and individual learner characteristics, but also context factors, such as, the knowledge domain, task requirements, or additional instructional support, influence the effectiveness of dynamic visualizations (e.g., Hoffler & Leutner, 2007; Lowe, Schnotz, & Rasch, 2011; Tversky, Morrison, & Betrancourt, 2002). These context factors have become a focus of research on dynamic visualizations. Learning with Gestures One idea on how to support learning about movements with dynamic visualizations that is based on the embodied cognition approach and proposed by De Koning and Tabbers (2011) is the active and passive use of gesture. Empirically, Hegarty et al. (2005) showed that gestures are naturally used to express movements of depicted components and thereby also the depicted processes in mental animation problems. Moreover, it has already been shown that the production of gestures during learning is beneficial for acquiring knowledge about different scientific topics and spatial problem solving (e.g., Chu & Kita, 2011; Cook & Goldin-Meadow, 2006; Scheiter et al., 2012). However, learners can either produce gestures on their own or they can perceive gestures that are performed by others. In line with the proposal of De Koning and Tabbers (2011), it is also beneficial for learning to perceive gestures that illustrate the depicted contents, for instance, performed by teachers (e.g., Valenzeno, Alibali, & Klatzky, 2003). Underlying this gesture watching effect might be the activation of brain areas (i.e., the human mirror-neuron- system [hMNS]; Fogassi & Ferrari, 2011; Rizzolatti & Craighero, 2004) that are typically used to observe, understand and imitate the actions of other persons. In a related line of research, a current hypothesis that has recently received considerable attention (e.g., Ayres et al., 2009; Van Gog et al., 2009) is that the stimulation and involvement of this hMNS might be beneficial for learning about complex continuous aspects with dynamic visualizations. The hMNS is typically activated by human movements, but may be more generally used to also represent other biological or even non-biological movements, if the observer is able to anthropomorphize