# Necessary Morphological Patches Extraction for Automatic Micro-Expression Recognition
## 1 Introduction  
Micro-expression is a brief, involuntary, and external representation of a real emotion that can be exploited to determine the \"real\" behaviors and feelings of an individual [1]. Micro-expression was first discovered by psychologists [PERSON] and [PERSON] in 1969 [2]. Compared with ordinary facial expressions, micro-expressions have three significant characteristics: short duration, (generally lasts \(1/25\) to \(1/3\) s), low intensity, and (usually) local movement [3]. Based on these characteristics, micro-expressions are very difficult to detect by human beings [4]. Only highly trained individuals may be able to distinguish them, but even with proper training, the accuracy of their recognition falls below \(50\%\) in general [5]. However, it is very crucial to have the capability to detect and recognizemicro-expressions in many areas, such as psychological and clinical diagnosis, police interrogation, and national security [6]. Up till now, there have been plentiful research works for this area in the literature [7; 8; 9; 10; 11; 12; 13], in which computer-aided techniques have been established to automatically recognize micro-expressions. According to the known research results [8; 9; 10], extracting features from the whole facial region can be found in most experiments, thus generating a lot of unnecessary redundant features so that the efficiency of recognition is greatly reduced. Psychologists have also found that micro-expressions tend to be partial movements that do not appear in the upper and lower portions of the facial region simultaneously [5] and are generally concentrated near the region of eyes, nose, and mouth. [PERSON] proposed that facial expression represents slight changes in several discrete facial motion units [5]. He also established Facial Action Coding System (FACS) [14] to describe the relationship between facial muscle changes and emotional states. This system illustrates that facial expressions are associated with subtle changes in several action units (AUs). Compared with ordinary expression, micro-expressions have extremely brief facial representations and invoke less muscle action units. For example, even without the involvement of eyebrows and mouth, one can express their inner state of surprise only by raising the upper eyelids. Later on, the theory of necessary morphological patches (NMPs) for micro-expression has established [15]. NMPs denote that some regions are indispensable in micro expression. For example, in all facial representations of disgust, the eyebrows and mouth do not need to be raised and open respectively, but the upper lip must be raised and appearing nasolabial sulcus on both sides of the nose. Figure 1 shows the NMPs of disgust, which are the indications of micro-expression to judge whether a person has a disgust emotional state. Almost all emotional information of micro-expressions is concentrated in these patches, therefore we need to separate these patches from the whole facial area. In this paper, we aim to find some NMPs that play a key role in micro-expression recognition, and use these patches to train and learn how to identify micro-expression sequences. Automatic facial landmark detection is the first step in this work. This technique can detect the face in the video sequences, and adjust the position of the face in order to cut out the facial region. Face alignment is applied to extract facial active patches. In this study, in order to locate the facial active patches more accurately, we chose the algorithm of 68 landmarks [10]. After finding these patches in the face, we calibrated the active patches of the eyebrows, eyes, nose, and mouth based on the FACS criterion and landmarks technology. Later, we manually cut out 18 active patches, known as regions of interest (ROI) [13] in the whole face area. Moreover, we need to extract more effective features from these active patches [16; 17; 18; 19].