# Motivation and emotion/Book/2011/Facial expression

* Multimedia presentation (5 min)

## Overview

How do you react when you see someone smiling? Do you smile back? React warmly towards that person? Or do you become angry and turn away? How is that we know to interpret a lift of the upper lip as a positive sign? Facial expressions, which often occur involuntarily, are social tools critical for the development and regulation of interpersonal skills. Facial expressions refer to movements or adjustments to the facial muscles to produce universally recognised displays of emotion. Humans are born with an innate and effective way of decoding such emotional facial expressions, such as a lift of the upper lip (smile) indicating happiness. However, humans, unlike many other kinds of species, are also born with the ability to hide emotions and be deceptive to others. Fortunately (or unfortunately?), researchers have discovered many characteristics which can suggest that voluntary expressions are being produced.
These ideas are well depicted in the popular American television program Lie to Me, which is centered around a “human polygraph machine” known as Dr. Carl Lightman who, along with his team of trained deception experts, specialises in solving criminal cases by reading the emotion filled facial expressions of the suspects. The show is based on the work of leading researcher in the field of facial expressions, Paul Ekman.
The use of emotional expression as a way of communication of emotions or information has always almost existed for humans. In pre-linguistic times, humans used facial expressions to convey important information such as danger, affection, or the sight of food. However, it is only now that scientists have become to understand how and why we communicate emotions through facial expressions.

## Influential Theorists

The history of research on emotional facial expression is not a long one. Currently, research goes back only as far as Charles Darwin, who in 1872 authored the evolutionary book Expression of Emotion in Man and Animals. Since then, only a handful of researchers have made significant contributions in the field of emotional expression. Dr. Paul Ekman, is credited as leading researcher in the field today, building on the works of Duchenne De Bologne, Darwin, and collaborating with researchers such as Wallace Freisen to produce theoretical concepts and models to further our understanding of facial expressions.

### Charles Darwin

Charles Darwin is renowned for his evolutionary contributions to psychology, introducing concepts such as survival of the fittest and natural selection to the non-scientific world. What isn’t so well-known about him is the contribution he made to understanding emotional facial expression in humans and nonhumans. In his influential book, Expressions of Emotion in Man and Animals, Darwin draws focus to select emotions which are communicated through discrete facial movements across different cultures and species (Darwin, 1872). Darwin emphasises the process of evolution in facial expressions. He suggests that facial expressions represent adaptive features that were once functional in the past (or still are today) for either human or nonhuman species. For example, eyebrow raising (a characteristic of the emotion of surprise) increases the field of vision and allows the eyebrow to move in any direction, a highly functional action (Matsumoto & Ekman, 2007).

### Paul Ekman

Unquestionably, the most influential researcher in the field of emotional expression this century is Dr. Paul Ekman. Ekman’s infatuation with understanding facial expressions began when he was teaching a young group of psychiatrists. The group wanted to know how to tell if a psychiatric patient was telling the truth about feeling better, a question which fueled intense research by Ekman (Henley, 2009). Eventually, Ekman discovered, whilst watching videos of dishonest psychiatric patients in slow motion, that vivid intense emotions flashed for 1/5th of second on the patients face, leading to the discovery of micro-expressions- "very fast, intense expressions of concealed emotion" (Henley, 2009). Over the past two decades, Ekman has been the front man in research based on understanding how and why humans communicate emotions through emotional expression. His extensive research lead to the discovery that emotional expressions are universal (Ekman, 1993), and the development of a core set of emotions (Ekman, 1992b), both of which will be discussed in more detail in the following section. Along with fellow researcher Wallace Friesen, Ekman developed the internationally renowned Facial Action Coding System (Ekman & Freiesen, 1978).

#### Facial Coding System

Along with fellow researcher, Wallace Friesen, Ekman co-created the Facial Action Coding System (FACS)(1978) a system for “describing all visually distinguishable facial movements” (Ekman & Freiesen, 1978). The system is based on the movement of action units (AU’s) which are anatomically related to distinct head and eye distortions and reflect a particular facial expression (Levenson & Friesen,1983). FACS consists of 46 action units, which produce around 7,000 different combinations, providing the necessary detail for describing facial expressions. For example, according to FACS, the expression of happiness involves a combination of pulling lip corners (AU 12 and 13), mouth opening (AU 25 and 27), upper lip raising (AU 10), and often burrow deepening (AU 11) (Sayette, Cohn, Wertz, Perrott, & Parrot, 2001). Today, FACS is used as an effective means of measuring facial expression, often used as a deception tool, showing the difference between a genuine and fake expression. Since its creation over 30 years ago, FACS has been updated to computer software, making the process faster and more accurate and highly accessible.

### Alan Fridlund

Although the work of Darwin, Duchenne, Ekman and Freisen has gathered a lot of support, as with any scientific domain, opposing theorists still exist. Alanh Fridlund proposes what is known as the behavioural ecology theory which depicts facial expressions to be reflection of intentions rather than emotions (Fridlund, 1994) . Fridlund believes that facial expressions are purely communicative signals linked to underlying emotional states. Fridlund based his theory on evolutionary grounds. He reasoned that facial displays could only evolve if they were noticed by others, essentially, facial expressions cannot evolve without social interaction (Azar, 2000). Although both Ekman and Fridlund have persuasive arguments, it is generally accepted in the scientific world that facial expressions are a product of both communicative and emotional functions.
Test Yourself: Section Review Question

## Theoretical concepts

[Provide more detail]

### Universal nature of facial expressions

Early thoughts in the scientific world was that facial expressions, like language, were culture specific (Bruner and Tagiuri, 1954). However, in the late 1960’s Ekman and Tomkins began the first of many studies to show that facial expressions are universal. In the early studies, participants selected from a diverse range of cultures were required to match basic emotions with pictures of different facial expressions (Ekman, Sorenson, and Frisen, 1969). The result indicated that there were agreements across cultures in the emotion represented by the facial expression, and Ekman and colleagues concluded that this represented universality of emotional expression. At this stage, only six emotions- anger, disgust, fear, happiness, sadness and surprise- were found to be universal. However, soon critics began to question the validity of these studies, suggesting that the results might simply be due to shared visual input as all the participants used in the study where from industrialised cultures were emotions could be learned and imitated through media. In response to these claims, Ekman and Friesen studied a preliterate, isolated culture in New Guinea. They found that the tribespersons accurately identified facial expressions to match stories depicting the corresponding emotion (Ekman, & Friesen, et al, 1987). In addition, Ekman and Friesen filmed the faces of the tribesmen portraying a range of facial expressions, and showed these pictures to American participants. The American participants were able to correctly identify the emotions corresponding to the facial expressions portrayed by the New Guinea tribesmen, providing a source of evidence for universality of facial expressions, and dispelling concerns that previous results were due to shared visual input (Ekman, & Friesen, et al, 1987). However, following the publishing of these studies, critics again spoke out, this time with new concerns that the previous studies looked at judgments and not the production of facial expressions. In response, Ekman conducted another study in which he instructed Japanese and American participants to watch a emotion provoking film. He analysed the facial expressions of both participants, with results indicating that participants from both cultures displayed the same muscle movements in response to the emotional film (Matsumoto, & Ekman, 1989). Since these consecutive studies many other researchers have also demonstrated that there a number of universally recognised facial expressions (Matsumoto, 2001), and it is now generally accepted in the scientific community that facial expressions are not culturally dependent.
In addition to his research on the universality of facial expressions, Ekman also proposed the ideas of display rules to describe how different cultures teach its members different rules about the expression of emotions in social situations (Ekman, Friesen, & O’Sullivan, 1987). To show this, Ekman studied Japanese and American individuals and found that when the subjects were alone in a room there were no differences in facial expression in response to a emotional provoking film. However, when authority figures were also present in the experiment, the Japanese participants were more likely to attempt to hide negative expressions with a smile (Ekman, 1989). These cultural differences in displaying emotion may account for how on occasion some universal expressions might appear to differ within cultures.

### Innate nature of facial expressions

Facial expressions are essential for communication and social interaction at all ages, however, they are especially important during the first few years of life when children are verbally less able to communicate. There is consistent evidence for the ability of infants to not only discriminate emotional expression, but to also produce facial expressions similar to the ones show by adults (Izard, 1994). A study of emotion in young infants found that at 2.5 months of age infants are able to understand expressions of sadness and anger. In addition, majority of infants were able to display basic characteristics of surprise disgust, and fear (Izard, 1994). Izard (1994) suggests that these facial expressions are indeed accurate indicators of the emotion that the infant is feeling at the time. Furthermore, studies have also shown that children who are blind or who become blind at a young age are still able to express feelings by smiling, crying, and glaring, representing the emotions of joy, sadness, and anger respectively (Eibl-Eibesfeldt, 1973). Camras (1992) studied 5 month old Japanese and American infants and found that they produced identical expressions of anger, and also sad-anger and sad-fear combinations. This study, not only shows that humans are born with an innate tendency to display emotions through facial expressions, but also supports the universality theory of facial expressions.

### Facial feedback hypothesis

The Facial Feedback Hypothesis originated from the work of Darwin who was the first to propose that stimuli-evoked physiological changes have an impact on emotion, and are not just a consequence of emotion (Darwin, 1872). Fellow psychologist William James even went as far to say that emotion is the purely the awareness of physiological changes following an internal or external stimulus (Reisanzein, & Studtmann, 2007). Tomkins conceptualised these proposals to form the bases of the Facial Feedback hypothesis which suggests that facial expressions have the ability to bring about an emotional experience in an individual (Tomkins, 1962).
* “The face expresses affect, both to others, and to the self, via feedback, which is more rapid and more complex than any stimulation of which the slower moving visceral changes are capable” (Tomkins, 1962).
Lanzetta and colleagues (Lanzetta, Cartwright-Smith, & Kleck,1976) were one of the first to conduct experimental studies on the facial feedback hypothesis, finding that participants who were instructed to express an emotion had a tendency to feel that emotion more strongly. Lanzetta found that individuals who were instructed to fake a high pain threshold whilst receiving electric shocks experienced less physiological pain responses than those who were instructed to exaggerate the pain they were feeling. One well known study, which has since been replicated, involved participants holding a pencil in their mouths with either their lips (position which would contract facial muscles involved in a frown expression), or with their teeth (position that would contract the facial muscles involved in a smiling expression). The participants were then required to watch a cartoon and rate it’s comedic value. Results showed that individuals who held the pencil with their teeth (the smiling expression) rated the cartoon more favourably than those in the other condition (McIntosh, 1986).
Although this study, and along with many others, favour the Facial Feedback hypothesis, there are many others that contradict this theory. In particular, critics argue that individuals who suffer from permanent facial paralysis are still able to experience emotion (Fridlund, 1994). Recently, researchers have begun to look into the effects of Botox on facial expression, with some results indicating that temporary facial paralysis has only a select effect on the processing of emotional stimuli (Ochsner, 2010), opposing the facial feedback hypothesis.
Test Yourself: Section Review Question

## Deception

* “if his lips are silent, he chatters with his finger tips, betrayal oozes out of him at every pore” (Freud, 1905).
Body reactions, inconsistent speech patterns, and fidgeting are all commonly looked for signs of lying, however, what is often ignored are the many nonverbal facial signs of deception. In fact emotion is one of the main reasons why lies fail. Even when an individual is not lying about their emotions, there are many other emotions that occur as a result of lying such as; fear, guilt, and even excitement (Yuille, 1989). Darwin was one of the first to investigate the effect of deception on facial expression, by suggesting that actions which are cannot occur voluntarily are unable to be prevented when other involuntary processes such as emotion occur. In essence, Darwin believed that humans were unable to prevent their true emotions or feelings from being expressed facially (Darwin, 1872). In 1969, Ekman provided substantial evidence for this notion when he discovered micro-expressions- vivid intense emotions which last around 1/5 of a second- which although occur very briefly can be detected by the human eye with relevant training (Ekman, & Friesen, 1969). In addition to micro-expressions, Ekman believes there are seven other characteristics which help to distinguish whether a facial expression is occurring voluntarily or involuntarily:
* Morphology- the absence of any reliable movements can hint that the expression is voluntary
* Symmetry- asymmetry is an indication that the expression is dilberate
* Duration- very brief (less than 1/5 a second) and long (greater than 5 seconds)suggests a more deliberate than spontaneous response.
* Speed of onset- a deliberate expression will typically have a quicker onset than a spontaneous one (although this does tend to vary with social circumstances)
* Apex overlap- in expressions in which multiple facial actions are present, the apexes of the action will overlap in a voluntary expression.
* Ballistic trajectory- a voluntary expression will appear smooth over it's trajectory, without a stepped off or rough onset.
* Cohesion- the expression is spontaneous it will coincide with what is being said vocally (Ekman, 2003).
Another important behavioural cue is the distinction between enjoyment smiles and non-enjoyment smiles, as suggested by Duchenne de Boulogne (1862). It has been suggested that smiles when lying differ from involuntary smiles. One study found that woman who were instructed to conceal negative emotion and claim enjoyment displayed what is referred to as a masking smile, a superimposed expression which attempts to conceal muscular actions associated with fear, sadness, and disgust (Frank, Ekman, & Friesen, 1993). Although verbal information is very useful in determining whether a person is attempting to conceal information, Ekman has shown that nonverbal signs can also provide substantial evidence for deception.
Test Yourself: Section Review Question
Facial Expressions
In 1969, Ekman and Friesen (with the help of Silvan Tomkins) compiled a list of what they believed to be basic human emotions (Ekman, & Friesen, 1971). These emotions were selected based on research in different cultures, in particular illiterate cultures where facial expressions can’t be taught. Ekman deems this one characteristic of a basic emotion- it must be universal. Other characteristics of a basic emotions include; it must be associated with a physiological response, there must be universal events in which these emotions occur, and the emotion must be able to be triggered immediately without conscious awareness following a internal or external stimuli. (Dagleish and Power, 1999).
Below are the six basic emotions according to Ekman, and details of their corresponding facial expressions and facial muscles.
Star of Lie to Me, Tim Roth, demonstrating Ekman's six basic emotions
Fear
Fear occurs when an individual interprets a situation as dangerous or threatening, either physically or psychologically. When the fear response is activated, defense mechanisms
also respond. The mechanisms include; perspiration, trembling, increased heart rate and blood pressure (Marsh, Kozak, Ambady, 2007), and facial expressions such as: wide opened eyes, lips retracted, eyebrows raised, and mouth opened (Matsumoto & Ekman, 2007).
Fear: Facial Expression and Corresponding Muscle
Facial ExpressionFacial Muscle
eyes wideoccipito frontalis muscle
mouth openlevator labii muscle
eyebrows raisedfrontalis and corrugators muscle
lips retractedbuccinators muscle
Anger
Anger is one of the most uncomfortable emotions to experience. It often begins with a build up of other emotions such as intent, blame, or annoyance, and become expressed when an individual perceives a situation to be unsatisfactory often due to an interference of goals (Power & Dalgleish, 2008). Often anger results in acts of violence and aggression, and is best characterised by the following facial expressions: raised or flared nostrils, a furrowed brow, wide open eyes, compressed mouth, and head in an erect position (Matsumoto & Ekman, 2007).
Anger: Facial Expression and Corresponding Muscle
Facial ExpressionFacial Muscle
nostrils raisednasalis muscle
mouth compressedmentalis muscle
furrowed browprocerus muscle
eyes wideoccipito frontalis
Sadness
Sadness is an emotion that often occurs following experiences of separation or failure, and motivates an individual to initiate whatever behaviour necessary to cease the sadness provoking stimuli. Although the emotion of sadness is often only temporary, it is usually followed by prolonged feelings of sorrow and unhappiness (Reeve, 2009). Signs of sadness including crying and sobbing, and facial expressions such as depression of the corners of the mouth, and drooping of the eyelids (Matsumoto & Ekman, 2007).
Sadness: Facial Expression and Corresponding Muscle
Facial ExpressionFacial Muscle
depression of corners of mouthdepresser labii inferior muscle
drooping of eyelidsorbicularis oculi muscle
Disgust
Disgust can occur in response to a variety of different objects, situations, and experiences, and therefore is a very subjective emotion. According to Reeve (2009) however, most people tend to be disgusted by things of an animal origin that can potentially spread and infect others objects. In response to feelings of disgust, an individual becomes motivated to dispel the contaminated object, producing facial characteristics such as: lower lip turned down, upper lip raised, mouth open, and protruding lips (Matsumoto & Ekman, 2007).
Disgust: Facial Expression and Corresponding Muscle
Facial ExpressionFacial Muscle
raising of the upper liplevator labii muscle
lower lip turned downmentalis muscle
mouth openlevator labii muscle
Happiness
The term happiness can be referred to in terms of moods, experiences, feelings and emotions. Generally, it represents an emotional state which reflects satisfaction across many different domains of life. Happiness is often associated with other emotions such as joy, exhilaration, and ecstasy (Power & Dalgleish, 2008). It is most often experienced when reaching a goal, receiving feedback, or looking forward to an upcoming event, and is best represented by a smile- wrinkled skin under the eyes, raised check bones, and mouth drawn up at the corners (Matsumoto & Ekman, 2007).
Happiness: Facial Expression and Corresponding Muscle
Facial ExpressionFacial Muscle
mouth drawn up at cornerszyhomaticus major muscle
wrinkled skin under the eyesoccipito frontalis muscle
raising check bonesbuccinators muscle
Surprise
Surprise is the briefest of all emotions, often only lasting a few seconds, after which it is then transferred into other emotions such as fear, relief, emotion, or amusement. Surprise is an emotion which can only be triggered by an unplanned event, and can be described as either a pleasant or unpleasant emotion depending on both the situation and the individual (Power & Dalgleish, 2008). Surprise is a unexpected emotion which results in dramatic facial expressions including; raised eyebrows, wrinkled forehead, open wide eyes, and gaping mouth (emotions revealed TB) (Matsumoto & Ekman, 2007).
Surprise: Facial Expression and Corresponding Muscle
Facial ExpressionFacial Muscle
raised eyebrowsoccipito muscle
wrinkled foreheadfrontalis muscle
open wide eyesoccipito frontalis
gaping mouthlevator labii muscle
Test Yourself: Section Review Question
Point added for a correct answer:
Points for an incorrect answer:
Ignore the questions' coefficients:
Mouth drawn up at corners, wrinkled skin under the eyes, and raised check bones indicate which emotion?:
Surprise
Joy
happiness :
* Applications
* [Provide more detail]
* Autism
* Reading emotion filled facial expressions in others is key to social interactions as it helps to explain and anticipate other’s actions. Individuals who do not recognise these facial expressions are prone to misinterpreting information which can have detrimental effects on social interaction (Ekman, 1992). Typically, individuals with Autism have a deficit in interpreting, understanding, and responding to emotional expression (Celani et al, 1999). There are hundreds of theories to suggest why individuals with Autism struggle to understand and respond to emotional expression. One current theory, is that although individuals with Autism have the ability to perceive emotional expression, they do not give the same priority to interpreting facial expressions as typical individuals do (Celani et al, 1999). This is demonstrated in studies where participants with Autism, who are asked to pair pictures of people together, have a tendency to pair pictures on the basis of non-emotional features, such as a hat (Celani et al, 1999). Scientists in Singapore have developed a system which could potentially help Autistic individuals identify the emotion of others. The system first locates the outer edges of the face, then narrows it down to crucial features, and then classify those features into corresponding emotions (Teoh, Nguwi, Cho, 2009). This system is thought to help individuals to read the emotions in other through facial expressions, however, it does not teach them how to react to such emotions, a key deficit in individuals with Autism. Studies on individuals with Autism further deepens the understanding of facial expression in humans, and continual research in this area may potentially lead to effective management of emotion deficits in the Autistic population.