From Data Streams to Information Flow: Information Exchange in Child-Parent Interaction

From Data Streams to Information Flow: Information Exchange in Child-Parent Interaction Heeyoul Choi, Chen Yu, Linda B. Smith, and Olaf Sporns (chenyu@indiana.edu) Psychological and Brain Sciences, and Cognitive Science Program Indiana University, 1101 E. 10th St., Bloomington, IN 47405 Abstract learning the relevant sensorimotor cues and thus may enable us to see more clearly the strands that are more tightly woven in the highly developed adult system. The key idea of the present study is to apply information- theoretic measures to understand the structure in the sensorimotor dynamics of the interaction. To this end, we conceptualize multimodal information flows between children and parents as those between senders and receivers in artificial communication systems (Shannon, 1948). More specifically, the child and the parent communicate with each other using multiple communication channels such as gaze, pointing, speech, and hand movements. The specific goal this study is to understand how information theoretic measures might be used to analyze the flow and information exchange within each participant and between participants. For example, within an individual, do behaviors such as looking “send” information to the hands, in the sense of signalling a reach? Across individuals, does a hand action by one participant send information to the gaze of the other? And, if we can measure information flow in these ways, can we also measure how it might change at different points in the interaction, for example, when an object is being named? Historically, information theory was developed to find fundamental limits on compressing and reliably communicating data within single transmission channels (Shannon, 1948). Since its inception, information theory has found applications in many other areas, including statistical inference, natural language processing, the evolution and function of molecular codes, model selection in ecology, thermal physics and other forms of data analysis(de Ruyter van Steveninck et al., 1997). Recently, information theory has been applied in the context of embodied autonomous systems to help characterize the flow of information between (neural or algorithmic) control architectures, body and environment (Sporns & Lungarella, 2006). Despite the recent success of information-theoretic measures in various scientific fields, these recent advances have not been systematically applied to human behavioral data. Thus the present study seeks both to understand the sensorimotor dynamics of social interactions as information flow and to develop a mathematically rigorous framework within which to do so. The goal of this paper is to enhance understanding of how bodily actions between two social partners are coordinated in interpersonal interactions in naturalistic contexts. To this end, we introduce information-theoretic measures as a new approach to capturing sensorimotor dynamics in child-parent social interaction. In particular, information flows were measured based on a set of variables extracted from multimodal fine-grained behavioral data in social interactions wherein a child and a parent played with a set of novel toys. Our results showed that information-theoretic measures can indeed capture the inherent structure of perception and action dynamics and further information exchange patterns can be used to predict successful learning through child-parent interactions. Moreover, those information flows between sensorimotor variables reveal a set of underlying perceptual and motor patterns with cognitively plausible explanations. In summary, the present study represents the first steps to connect information-theoretic measures as a mathematically rigorous framework with embodied human communication and cognition. Keywords: Embodied Cognition; Local Transfer Entropy; Information Theory; Word Learning; Social Interaction. Introduction How do two interacting agents couple their activity? Some forms of human collaborative and coordinated behavior (such as maintaining a conversation, or jointly solving a complex problem) appear to happen effortlessly as if the participants can read each other’s mind and understand each other’s communicative intent. At an elementary level, inter- agent coordination depends on external (and observable) behaviors by the participants where the behavior of one participant influences the behavior of the other. Past research tells us that behaviors such as eye movements, head turns, and hand gestures are critical to this coordination. However, very little is known about the real- time dynamics of these behaviors in social interactions nor about how they may be related to higher-order functions such as making inferences about the goals and intentions of other. Because so little is known about the real time dynamics of the sensory-motor behaviors on which social coordination rests, the present study takes a bottom-up approach, measuring multiple sensory streams – head and hand movements as well as each participant’s view of the events – and then attempts to determine the possible signatures of coordination in these behaviors. The social coupling of a toddler and a parent is an appropriate first setting for this endeavor because the toddler as a developing system is just Experiment and Data Preprocessing Figure 1 provides an overview of the approach. We measured multiple sensory streams with no prior expectations that they are independent or dependent. These