Auditory, Visual, and Auditory-Visual Identification of Emotions by Nursery School Children (2012)

Perception of nonverbal information is essential for communication and social interaction because the full meaning of any communicative message is conveyed not only by its verbal content but also by nonverbal cues. The nonverbal part of the message relates to the speaker's mood, emotional state, and attitude toward the message, with components including facial expression, tone of voice, and rate of speech. Ability to perceive a message's nonverbal components is essential for appropriate, successful interpretation as well as for continuation of the social interaction (Denham, Zoller, & Couchhoud, 1994; Dunn, 1995; Feldman, Philippot, & Custrini, 1991). Nonverbal information on the emotional state of the speaker comprises both visual and auditory cues. Although people may differ in their sensitivity to emotion perception, research has demonstrated that accurate judgments of emotions and feelings can be made from wordless vocal messages (Hammermeister & Timms, 1989; Siegman, 1987; Wallbott & Scherer, 1986). The clearest, most consistent auditory factors in signaling a speaker's emotional state were found to be the mean value of the fundamental frequency [(F.sub.0]), its range, and its rate of changes. Duration of production and changes in voice intensity were also described as important parameters (Johnstone & Scherer, 2000; Scherer, 1982, 1986; Siegman, 1987). Sad utterances, for example, showed only small and slow changes in [F.sub.0], leading to a relatively flat contour shape along a long utterance. In contrast, the contour shape for angry utterances showed high [F.sub.0], with a considerably greater range of [F.sub.0] compared to the range for neutral utterance. Also, angry utterances demonstrated a higher intensity and rate of speech (Johnstone & Scherer, 2000; Scherer, 1986; Williams & Stevens, 1972). Visual transmission of emotions is considered to be highly efficient. Visual cues regarding emotions are clear, and observers can reliably identify different emotions based on them (Fridlund, Ekman, & Oster, 1987; Hess, Kappas, & Scherer, 1988; Most, Weisel, & Zaychik, 1993; Wallbott & Scherer, 1986). Researchers have suggested that anger, disgust, happiness, fear, sadness, and surprise are each expressed visually in a universally distinct way (Ekman, 1982). For example, fear is expressed by raised eyebrows, wide-open eyes, and open mouth, whereas sadness is expressed by knitted eyebrows, narrowed eyes, and downward-drawn mouth corners (Camras, Malatesta, & Izard, 1991). Research on adults' identification of emotions when cues were presented either auditorily or visually indicated that the visual modality always yielded superior identification compared to the auditory modality. When the stimulus was presented via the combined auditory-visual modality (AV), its identification surpassed the auditory modality alone but did not always significantly surpass the visual modality alone (Fridlund et al., 1987; Gross & Ballif, 1991; Hess et al., 1988; Wallbott & Scherer, 1986). Most and colleagues investigated the identification of emotions via the different modalities in adolescents with and without hearing impairments (Most et al., 1993) and in adolescents with and without learning disabilities (Most & Greenbank, 2000). They reported that adolescents in all of these groups perceived emotions least well through the auditory channel alone. Adolescents with normal hearing and those without learning disabilities also showed better perception of cues through the combined AV modality than through the visual modality alone. Likewise, adolescents with learning disabilities identified emotions better through the combined AV modality than the visual modality alone, although they made poorer use of the auditory cues provided in this modality when compared to their peers without learning disabilities. The adolescents with hearing loss, however, did not demonstrate the advantage of the combined modality in comparison to the visual modality in their identification of emotions. …