Multisensory Sensemaking: Children's Exploratory Behavior has Organizing Structure at Micro- and Macro-scales

Multisensory Sensemaking: Children’s Exploratory Behavior has Organizing Structure at Micro- and Macro-scales Nancy Owens Renner (nrenner@ucsd.edu) Department of Cognitive Science, University of California San Diego 9500 Gilman Drive, La Jolla, CA USA 92093-0515 Abstract • situated, i.e. physical, social, cultural, and historical contexts all play a role in cognition; • distributed, i.e. cognition at every scale manifests in nested systems with interacting parts. People live and act in cognitive ecosystems, in which brains, bodies, cultural and material worlds all exert mutual cognitive influence in any given moment and as they change over time (Hutchins, 2010). Distributed cognition provides a unifying principle. When children encounter museum exhibits, they find rich opportunities for action, perception, learning, and other forms of cognition. Can we see systematic organization in the children’s behavior, and by extension, their cognition? What would count as evidence of this organization? Based on an account of cognition as embodied, situated, and culturally mediated, this research illustrates how some cognition can be directly observed, manifested through interactions among modalities, people and objects in a distributed cognitive system. This field study uses micro- and macro-analyses of behavioral data recorded on video to discover organizing structure in children’s behavior and cognition, evidenced in allocation of visual and haptic attention, manipulation of objects, and use of written, spoken, and gestured language. At micro- and macro-scales, perceptual engagement with concrete objects precedes engagement with abstract concepts, as evidenced in motor behavior and content of speech and gesture. In this context, we describe learning as a process of adaptive coordination, rather than a product to be measured. Future studies will test observation-driven hypotheses related to development of perceptual skills and patterns of scientific thinking, promising relevance to educational practices. Research Methods To understand how the designed environment structures cognitive activity, we collected and examined data with several questions in mind: Can we see systematic organization in the children’s behavior, and by extension, their cognition? What would count as evidence of this organization? How do children move their bodies, use their senses, and utilize language to explore objects and engage with ideas? The research team collected video data over several days, including first-person perspective video, recorded with head-mounted cameras worn by elementary students, and third-person perspective video, recorded with hand-held cameras operated by researchers. These two perspectives provide a wealth of information about what children pay attention to, as well as their interactions in physical and social contexts. These two video data streams amount to approximately 16 hours of video, primarily focused on 10 participants (five girls, five boys, all bilingual), some for two visits to the exhibition, with supplemental video recorded at specific exhibits. Keywords: multimodal attention; distributed cognition; situated cognition; perceptual; conceptual; embodied learning; education Introduction To describe the complexity of learning involves an ecological model built of interactions among human brains and bodies, cultural practices and artifacts, all subject to evolutionary and developmental change over time. “Learning ecosystems” represents the prevailing perspective among educational researchers and cognitive scientists today (NRC, 2009). This ecological view stands in stark contrast to first generation cognitive science and folk models of learning. These focus on symbolic information processing, often with a disregard for the role of context, culture, and history, and ignoring cognition as a biological phenomenon that includes emotion, sensory-motor experience, and probabilistic models of learning through experience (Gardner, 1985). This study uses observational methods and defines cognition as: The research team indexed the video to create an outline of the event structure, making large-scale behavioral patterns and anomalies apparent for more detailed study. We transcribed and coded a subset of the video in a densely detailed manner (figure 1). The video coding system includes multiple dimensions that fall into broad categories related to modalities in coordination with physical objects, social interaction, speech, and reading behavior. The coded video permits a broad array of analyses, for example, duration and sequence of behaviors with individual exhibits, specific objects of attention (looking at and touching specimens, interactives, graphic panels), composition of social group, speech and gesture content, language (English or Spanish), reading, and behavioral configurations defined by multimodal coordination. • embodied, i.e. a body is required for cognition, and the nature of bodily experience profoundly shapes cognition; To answer the research questions, this study sought to describe a learning ecosystem and its organizational