Dynamical Emotions: Bodily Dynamics of Affect during Problem Solving

Dynamical Emotions: Bodily Dynamics of Affect during Problem Solving Sidney D’Mello (sdmello@memphis.edu) Institute for Intelligent Systems, University of Memphis Memphis, TN 38152, USA There are distinctive reasons for focusing on body movements over the face and speech. First, it could be argued that body motions are ordinarily unconscious, unintentional, and thereby not susceptible to social editing, at least compared with facial expressions, speech intonation, and some gestures. Second, human bodies are relatively large and have multiple degrees of freedom, thereby making the body an effective affective communicative channel. Third, the expectation of a systematic link between bodily movements and complex mental states is supported by embodied theories of cognition and emotion (Niedenthal, Most (but not all) of the studies that have investigated the role of body movements in the expression of affective states have focused on the degree of bodily arousal, specific postures (e.g., forward-leans, arms akimbo), and some gestures (e.g., pointing, hailing) (Bull, 1987; Coulson, 2004). Comparatively, little is known about the low-level bodily correlates of affective states like frustration, anxiety, and cognitive-affective amalgamations such as confusion and flow/engagement. This paper addresses this issue by analyzing how these affective states influence the dynamics of presumably unconscious bodily movement In addition to simple measures of bodily motion, such as amount of movement and variability in movement, we focus on complex system measures as an index of the embodied nature of cognition and affect. Though relatively well established in the cognitive sciences, dynamical systems perspectives are only beginning to gain momentum in the affective sciences (Camras & Shutter, 2010; Coan, 2010; Lewis, 2005). Dynamical theories of emotion challenge the classical view that a central affect program coordinates the physiological, behavioral, and subjective components of an emotional episode (i.e., an anger circuit (Ekman, 1992)). Instead, they posit that these components are loosely coupled and are constantly interacting in a self-organizing fashion. When this system is perturbed, it is jolted from its state of equilibrium, until it spontaneously reorganizes and equilibrium is restored. The emotion “emerges” from the attractor (a set of states towards which the system regularly converges to) in which this dynamical trajectory converges (Camras & Shutter, 2010). The embodied nature of cognition and affect lead us to hypothesize that variations in affective states will be associated with meaningful shifts in fluctuations of bodily movements. The present study addresses this hypothesis by showing that when individuals engage in complex problem solving tasks, patterns of bodily movement exhibit dynamic signatures that may give away whether someone is in an anxious state, a confused state, a frustrated state, etc. Abstract This paper investigates the low level bodily correlates of affective states, such as boredom, confusion, anxiety, and frustration, that spontaneously emerge during complex problem solving tasks. Participants were video recorded while they solved difficult analytical reasoning problems after which they self-reported their affective states via a retrospective affect-judgment protocol. Time series of bodily motions were automatically extracted from the videos of participants’ faces and upper bodies via a motion filtering algorithm. Recurrence quantification analyses revealed that participants who reported increased levels of anxiety and frustration had less recurrent and deterministic movements compared to their counterparts. Importantly, these patterns could not be explained by the mere amount of movement or the variability in movement, but by non-obvious dynamical patterns in movement. We orient our findings towards theories that emphasize complex systems approaches to studying emotion. Keywords: emotion; bodily movement; dynamical systems; recurrence quantification analysis. Introduction The significance of non-verbal channels in human-human communication is widely acknowledged, however, the impetus has been on oculesics (eye contact), peripheral physiology, facial expressions, gesture, and paralinguistic features of speech. Comparatively, little attention has been directed towards the role of gross body movement (i.e. posture, movement) as a non-verbal communicative channel. This oversight is unfortunate because it has been known for several decades that posture is an important diagnostic cue of emotions, attitudes, social relationships, personality, and a host of other affective, cognitive, and social constructs (Ekman, 1992; Oullier & Basso, 2010; Russell, Bachorowski, & Fernandez-Dols, 2003). When it comes to affective states (general term covering moods and emotions), which are the focus of this paper, it has been claimed that the face plays a primary role in discriminating between specific emotions such as anger and fear (Ekman, 1984). Posture and body movements, on the other hand, are sometimes considered to be a mere indicator of the intensity of an emotion, thereby being downgraded in importance as an affective communicative channel (Bull, 1987). This view has been subsequently challenged by a number of studies, which showed the efficacy of the whole body in communicating specific emotions and attitudes, independent of facial expressions and paralinguistic features of speech (Castellano, Mortillaro, Camurri, Volpe, & Scherer, 2008; Coulson, 2004; Scherer & Ellgring, 2007).