# Animation Expression Control Based on Facial Region Division

Authors: Liangjun Zhang
Journal: Scientific Programming
 (2022)

---

## Abstract

Science and technology are developing rapidly in the twenty-first century. With the development of information technology, computers play a great role in people’s life. At present, with people’s increasing love for animation, exquisite and realistic animation has become people’s pursuit goal. Generally speaking, the most impressive thing in animation is the animation character expression. Nowadays, with the rapid development of science, it is necessary to develop a computer technology that can be used in animation expression control technology. The facial division is just met by computer technology. It is very important for the animation to create an animation expression consistent with the character’s face. The character face has diversity and uniqueness, which plays an important role in animation expression control. There are many factors in the character’s face area that affect animation expression control. The coordinated movement of multiple facial organs shows various emotional states through the changes in muscle movements in various areas of the face, such as eye muscles, facial muscles, and oral muscles. It has strong integrity and particularity and has relatively high technical requirements. Generally, the expression control technology can transform and deform a specific area of the face. Based on the division of the face area, the computer technology is used to calculate the different expression features of the face for recognition, showing a more exquisite and realistic animation expression. Under this background, this paper divides the facial region and introduces the physiological structure of the face and the relationship and influence between facial expression and animation expression control. Several algorithms used in facial structure feature point extraction are compared experimentally. After experimental comparison, it is found that the improved algorithm is much more efficient than the original algorithm in the process of extracting facial feature points, can remove redundancy, greatly reduce the amount of operation data, and lay a good foundation for the follow-up animation expression control technology.


---

## Body

## 1. Introduction

In recent years, people have higher and higher requirements for the quality of life, especially for entertainment. In this social context, the development space of the animation industry is becoming larger and larger. Its forms are diverse, and its development is changing day by day. It occupies an irreplaceable important position in people’s life. While promoting social needs, the animation industry is a global industry that promotes exchanges between different cultures. As a new technology in animation production, animation expression control technology [1] gets rid of the traditional dull and complex animation production mode [2] and can be produced simply and quickly through computer technology [3]. The animation expression control technology realized by computer technology can control and adjust the facial expression of animation characters [4]. This technology provides a new idea for animation expression production [5]. In order to provide a better visual experience for the audience and make the expression of the animated characters more vivid, the facial expression is combined with the animated scene. This technology can greatly improve the efficiency of animation creation and reduce the large number of repeated drawing work in the creation process.Animation expression [6], as an important branch of computer animation research, is an insurmountable problem to make animation expression as realistic and vivid as a human face. Facial expression [7] plays a very important role in people’s daily communication. People have similar structures and the same feature set [8]. Facial region segmentation [9] has an important impact on animation expression control. When the nose and mouth on the face change, the two eyes will also change; The change from one face to another, especially the expression of emotion and words, is different [7] When expressing emotions again, the coordinated movement of multiple facial organs has its own characteristics [10]. When expressing feelings [11], we must pay attention to the priority [12] of facial features [13]. Eyes, mouth, eyebrows, and eyelids have higher priority, but chin, nose, tongue, ears, and hair have lower priority. In addition, although the range of head movement is very small, it is also an important feature of distinctive expression. In the research of expression animation, the generation of several representative expressions is mainly considered [14]. In order to convert the whole face or specific area of the face image [15], experimental research shows that very good results can be achieved by using the inherent space method [16] to calculate the features of various expressions [17]. Based on the method of facial feature points [18], the facial features are classified and processed. By adjusting the deformation factor [19], time parameter curve of relevant expression units, the animation process of facial expression is effectively controlled, so as to realize the facial expression control of animated characters. The animation expression produced by the method of animation expression control through facial region division [20] is vivid and realistic, which can achieve an ideal animation effect.


## 2. The Meaning of Facial Segmentation and the Principle of Animation Expression Control

### 2.1. Influence of Facial Segmentation on Animation Expression Control

Character facial expression animation is the most complex of all the structures to realize animation, which involves not only simple animation methods. In order to achieve a more vivid effect, the design of characters’ facial expressions involves research in many fields. Because the facial expressions of the characters are very rich, they are roughly divided into joy, anger, sadness, and joy, but the degree of expression is different, and each has its own expression method. The human facial structure is mainly composed of the skull and facial muscles. The skull mainly determines the structure and shape of the whole face, while the control of facial muscles determines the deformation of the whole face. The changes in our facial expressions such as joy, anger, sadness, and joy are produced by the movement of facial muscles. Therefore, according to the local characteristics of facial expression caused by facial muscle movement, the facial region is generally divided into seven regions. In fact, when facial expressions occur, they are not evenly expressed on the whole face. A large number of observations show that they are mainly expressed in the areas of the mouth, nose, eyes, and eyebrows.Psychological research shows that the expression of different emotions has different facial regions. For example, the surprise is mainly reflected in the movement of mouth muscles and eye muscles, as well as the changes of eye muscles and corner radian; For example, fear is mainly manifested in the closure of the mouth and eyes, the wrinkle of the whole facial contour, and so on. Therefore, when studying the change of animation expression, it is necessary to correctly identify the facial region related to this expression and correctly remove the regions not related to this expression, so as to achieve the ideal effect of animation. According to the research, the face is generally divided into blocks according to its physiological characteristics to simplify a complex and irregular surface, which provides the basis for animation expression control technology.A complete face recognition system includes three main modules. Firstly, input the image sequence to be detected, detect the face image through the face car inspection module, extract the expression feature, recognize it, and enter the final classification as shown in Figure1.Figure 1
Flow chart of the facial expression recognition system.


#### 2.1.1. Animation Expression Control Principle

When designing different animation characters’ expressions, the regional changes of each character are very different every time they express their emotions; there are also different emotional characteristics between characters. The most important parts of the face to express emotion are eyes, eyebrows, nose, and mouth. These parts are driven by the data of the expression-driving unit to achieve the combination of different expressions. The facial animation expression is divided into three driving modules: feature location, expression driving, and expression animation; then, the expression animation studied is mainly divided into three parts: eye blinking, eyebrow movement, and mouth opening and closing. As shown in Figure2, the expression system of face animation is briefly described.Figure 2
Block diagram of facial animation expression system.Then, according to this design idea, the system principle of face animation is proposed. Firstly, the face image is input in the face feature location module, and the input image is simplified by using the commonly used image processing method of the computer. Then, the face is located by using the method of the active shape model to find the feature points. This module prepares for the next module processing; next, the facial animation expression module is processed: module 1 is automatically matched from ASM to the new model; finally, face animation is processed as shown in Figure3.Figure 3
Schematic diagram of the face animation system.In the schematic diagram of the face animation system, the animation system is divided into three modules. In the specific implementation process, the most important modules are modules 1 and 2. Firstly, input the image data in module 1, find the facial feature part, and use ASM to model to match the expression data unit to module 2. The face model is established in module 2. The face model forms the face animation through the movement of the three most basic areas, namely eyebrow blinking, eye blinking, and mouth movement. Finally, the animation is generated.


### 2.2. Application of Animation Expression Control Technology

#### 2.2.1. Animation Deformation Control Technology Based on MPEG-4

As early as the twentieth century, character animation based on MPEG-4 began to appear. Taking character animation parameters as the driving standard, we found the feature points on the character animation face model and made a new modification to the position information through calculation, so as to form the latest facial animation expression. Usef to represent the corresponding feature point displacement data and use d, u, and D to represent the corresponding offset as follows:(1)f=DU∗d.


#### 2.2.2. Animation Deformation Control Technology Based on PCA

The principal component analysis is mainly used to reduce the dimension of features. If the number of data features is very large, we can think that only some of them are really interesting and meaningful, while other features are either noise or redundant with other features.PCA technology is a diversified statistical analysis method. It puts the original high-dimensional space into the low-dimensional vector space through a special vector matrix, which has relative independence. Finally, the most reasonable feature points are extracted;F is taken as the statistical vector of various emotion parameters; and Fav represents the average value of the vector:(2)F=Fav+k⟶E,F=GEm=Fav+gEmE.Based on the above two formulas, the following equation is established:(3)kr⟶=Fr−FavER−1,where generalized inverse matrix (right inverse) is defined as follows:(4)ER−1=ETEET−1.The new position coordinates of the feature point can also be transformed through the new feature pointf as follows:(5)P=P0+f∗U∗d,where P0 is the coordinate of the feature point under ordinary neutral expression, u is the FAPU related to the feature point, and d is the unit vector.By recording nine emotions including fear, surprise, and happiness, the information of the horizontal direction of the marker point is obtained, and the eigenvalue, contribution rate, and cumulative contribution rate of each main component are obtained, as shown in Table1.Table 1
Eigenvalue, contribution rate, and cumulative contribution rate of main components.
λ1λ2λ3···λ18Characteristic value9.62055.02451.5894···−0.0000Contribution rate0.53640.25480．0598···−0.0000Cumulative contribution rate0.53540.89440.9546···1According to the expression method of the above feature quantity, the facial animation expression is controlled to make the animation expression more vivid.


## 2.1. Influence of Facial Segmentation on Animation Expression Control

Character facial expression animation is the most complex of all the structures to realize animation, which involves not only simple animation methods. In order to achieve a more vivid effect, the design of characters’ facial expressions involves research in many fields. Because the facial expressions of the characters are very rich, they are roughly divided into joy, anger, sadness, and joy, but the degree of expression is different, and each has its own expression method. The human facial structure is mainly composed of the skull and facial muscles. The skull mainly determines the structure and shape of the whole face, while the control of facial muscles determines the deformation of the whole face. The changes in our facial expressions such as joy, anger, sadness, and joy are produced by the movement of facial muscles. Therefore, according to the local characteristics of facial expression caused by facial muscle movement, the facial region is generally divided into seven regions. In fact, when facial expressions occur, they are not evenly expressed on the whole face. A large number of observations show that they are mainly expressed in the areas of the mouth, nose, eyes, and eyebrows.Psychological research shows that the expression of different emotions has different facial regions. For example, the surprise is mainly reflected in the movement of mouth muscles and eye muscles, as well as the changes of eye muscles and corner radian; For example, fear is mainly manifested in the closure of the mouth and eyes, the wrinkle of the whole facial contour, and so on. Therefore, when studying the change of animation expression, it is necessary to correctly identify the facial region related to this expression and correctly remove the regions not related to this expression, so as to achieve the ideal effect of animation. According to the research, the face is generally divided into blocks according to its physiological characteristics to simplify a complex and irregular surface, which provides the basis for animation expression control technology.A complete face recognition system includes three main modules. Firstly, input the image sequence to be detected, detect the face image through the face car inspection module, extract the expression feature, recognize it, and enter the final classification as shown in Figure1.Figure 1
Flow chart of the facial expression recognition system.


### 2.1.1. Animation Expression Control Principle

When designing different animation characters’ expressions, the regional changes of each character are very different every time they express their emotions; there are also different emotional characteristics between characters. The most important parts of the face to express emotion are eyes, eyebrows, nose, and mouth. These parts are driven by the data of the expression-driving unit to achieve the combination of different expressions. The facial animation expression is divided into three driving modules: feature location, expression driving, and expression animation; then, the expression animation studied is mainly divided into three parts: eye blinking, eyebrow movement, and mouth opening and closing. As shown in Figure2, the expression system of face animation is briefly described.Figure 2
Block diagram of facial animation expression system.Then, according to this design idea, the system principle of face animation is proposed. Firstly, the face image is input in the face feature location module, and the input image is simplified by using the commonly used image processing method of the computer. Then, the face is located by using the method of the active shape model to find the feature points. This module prepares for the next module processing; next, the facial animation expression module is processed: module 1 is automatically matched from ASM to the new model; finally, face animation is processed as shown in Figure3.Figure 3
Schematic diagram of the face animation system.In the schematic diagram of the face animation system, the animation system is divided into three modules. In the specific implementation process, the most important modules are modules 1 and 2. Firstly, input the image data in module 1, find the facial feature part, and use ASM to model to match the expression data unit to module 2. The face model is established in module 2. The face model forms the face animation through the movement of the three most basic areas, namely eyebrow blinking, eye blinking, and mouth movement. Finally, the animation is generated.


## 2.1.1. Animation Expression Control Principle

When designing different animation characters’ expressions, the regional changes of each character are very different every time they express their emotions; there are also different emotional characteristics between characters. The most important parts of the face to express emotion are eyes, eyebrows, nose, and mouth. These parts are driven by the data of the expression-driving unit to achieve the combination of different expressions. The facial animation expression is divided into three driving modules: feature location, expression driving, and expression animation; then, the expression animation studied is mainly divided into three parts: eye blinking, eyebrow movement, and mouth opening and closing. As shown in Figure2, the expression system of face animation is briefly described.Figure 2
Block diagram of facial animation expression system.Then, according to this design idea, the system principle of face animation is proposed. Firstly, the face image is input in the face feature location module, and the input image is simplified by using the commonly used image processing method of the computer. Then, the face is located by using the method of the active shape model to find the feature points. This module prepares for the next module processing; next, the facial animation expression module is processed: module 1 is automatically matched from ASM to the new model; finally, face animation is processed as shown in Figure3.Figure 3
Schematic diagram of the face animation system.In the schematic diagram of the face animation system, the animation system is divided into three modules. In the specific implementation process, the most important modules are modules 1 and 2. Firstly, input the image data in module 1, find the facial feature part, and use ASM to model to match the expression data unit to module 2. The face model is established in module 2. The face model forms the face animation through the movement of the three most basic areas, namely eyebrow blinking, eye blinking, and mouth movement. Finally, the animation is generated.


## 2.2. Application of Animation Expression Control Technology

### 2.2.1. Animation Deformation Control Technology Based on MPEG-4

As early as the twentieth century, character animation based on MPEG-4 began to appear. Taking character animation parameters as the driving standard, we found the feature points on the character animation face model and made a new modification to the position information through calculation, so as to form the latest facial animation expression. Usef to represent the corresponding feature point displacement data and use d, u, and D to represent the corresponding offset as follows:(1)f=DU∗d.


### 2.2.2. Animation Deformation Control Technology Based on PCA

The principal component analysis is mainly used to reduce the dimension of features. If the number of data features is very large, we can think that only some of them are really interesting and meaningful, while other features are either noise or redundant with other features.PCA technology is a diversified statistical analysis method. It puts the original high-dimensional space into the low-dimensional vector space through a special vector matrix, which has relative independence. Finally, the most reasonable feature points are extracted;F is taken as the statistical vector of various emotion parameters; and Fav represents the average value of the vector:(2)F=Fav+k⟶E,F=GEm=Fav+gEmE.Based on the above two formulas, the following equation is established:(3)kr⟶=Fr−FavER−1,where generalized inverse matrix (right inverse) is defined as follows:(4)ER−1=ETEET−1.The new position coordinates of the feature point can also be transformed through the new feature pointf as follows:(5)P=P0+f∗U∗d,where P0 is the coordinate of the feature point under ordinary neutral expression, u is the FAPU related to the feature point, and d is the unit vector.By recording nine emotions including fear, surprise, and happiness, the information of the horizontal direction of the marker point is obtained, and the eigenvalue, contribution rate, and cumulative contribution rate of each main component are obtained, as shown in Table1.Table 1
Eigenvalue, contribution rate, and cumulative contribution rate of main components.
λ1λ2λ3···λ18Characteristic value9.62055.02451.5894···−0.0000Contribution rate0.53640.25480．0598···−0.0000Cumulative contribution rate0.53540.89440.9546···1According to the expression method of the above feature quantity, the facial animation expression is controlled to make the animation expression more vivid.


## 2.2.1. Animation Deformation Control Technology Based on MPEG-4

As early as the twentieth century, character animation based on MPEG-4 began to appear. Taking character animation parameters as the driving standard, we found the feature points on the character animation face model and made a new modification to the position information through calculation, so as to form the latest facial animation expression. Usef to represent the corresponding feature point displacement data and use d, u, and D to represent the corresponding offset as follows:(1)f=DU∗d.


## 2.2.2. Animation Deformation Control Technology Based on PCA

The principal component analysis is mainly used to reduce the dimension of features. If the number of data features is very large, we can think that only some of them are really interesting and meaningful, while other features are either noise or redundant with other features.PCA technology is a diversified statistical analysis method. It puts the original high-dimensional space into the low-dimensional vector space through a special vector matrix, which has relative independence. Finally, the most reasonable feature points are extracted;F is taken as the statistical vector of various emotion parameters; and Fav represents the average value of the vector:(2)F=Fav+k⟶E,F=GEm=Fav+gEmE.Based on the above two formulas, the following equation is established:(3)kr⟶=Fr−FavER−1,where generalized inverse matrix (right inverse) is defined as follows:(4)ER−1=ETEET−1.The new position coordinates of the feature point can also be transformed through the new feature pointf as follows:(5)P=P0+f∗U∗d,where P0 is the coordinate of the feature point under ordinary neutral expression, u is the FAPU related to the feature point, and d is the unit vector.By recording nine emotions including fear, surprise, and happiness, the information of the horizontal direction of the marker point is obtained, and the eigenvalue, contribution rate, and cumulative contribution rate of each main component are obtained, as shown in Table1.Table 1
Eigenvalue, contribution rate, and cumulative contribution rate of main components.
λ1λ2λ3···λ18Characteristic value9.62055.02451.5894···−0.0000Contribution rate0.53640.25480．0598···−0.0000Cumulative contribution rate0.53540.89440.9546···1According to the expression method of the above feature quantity, the facial animation expression is controlled to make the animation expression more vivid.


## 3. Facial Features and Facial Region Division

### 3.1. Facial Features

There are about 7.8 billion people in the world. Everyone’s face looks different from others and has their own unique facial features. It is precise because people can distinguish different faces by relying on different facial features. Although each face is different and has its own characteristics, there are many kinds of the same movement rules in the face. For example, the expression of the face is the same, and the movement rules of each area of the face and internal muscle groups are the same; For example, the positional relationship of facial regions or organs will not change. According to the basic features of the face, the aforementioned “three courtyards and five eyes” are shown in Figure4.Figure 4
Overview of facial features.


### 3.2. Facial Region Segmentation

In the research of facial region segmentation, the initial researchers only regard the face as an overall structure to establish an animated character model, not specific to a certain region of the face. If you want to realize the expression change of a certain part or an organ or deal with the eyes, nose, or mouth separately, you will find that this implementation is very difficult. Staff requires advanced and complex technical requirements. Therefore, at first, the application of this model has certain limitations, and it cannot achieve the ideal effect. Later, in order to solve this problem, researchers proposed a 3D face machine model based on face segmentation. The biggest feature of this model is that it no longer treats the face as a whole but divides the whole face into multiple block structures according to a certain law. Each block structure is modeled and processed separately. Each block structure is independent, interdependent, and inseparable, so as to realize the structural mode of low cohesion and high coupling. Finally, these individually processed block structures can be combined again to form complete and different facial expressions. Although, on the surface, facial expressions look different, the face has many of the same properties, such as the basic proportional relationship of the three courtyards and five eyes of the face and the positional structure relationship of the face nose, eyes, and mouth, which will not change due to the difference of the face or the change of expression. Therefore, the researchers found that this provides the possibility for facial region segmentation or can be used as the basis of segmentation. The advantage of this model is that the face segmentation will not destroy the integrity of the face and the relative position and proportion relationship between various parts or organs and will not change due to the segmentation. In the process of modifying from a general face geometric model to a specific face geometric model, the face surface will not be seriously deformed. The animation provides great convenience for the production of facial expressions.At the same time, the researchers have shown in the experiment that the face is an elastomer. If the model is not adapted at the beginning and if a point on the model is moved arbitrarily during the interactive modification of the ordinary facial geometric model, it is necessary to calculate the position change of the vertices of the whole facial model. The amount of calculation required by this method is huge and complex, and in practical application, it wastes humans, materials, and resources. If the facial expression is divided into several small blocks, it is stipulated at the beginning that the motion of each point only affects other points in the block where the point is located, not the points of other blocks. In this way, it will simplify the complex and huge amount of calculation, significantly reduce the amount of calculation, and improve the accuracy and efficiency of calculation. Therefore, it is very necessary and effective to divide the face into regions.After the face is divided into blocks, in order to operate and process the geometric model of the face, it is necessary to design a good data structure for this geometric model. This data structure needs to meet the following requirements: there is a separate data block; this data block can be stored separately for the segmented organs or regions of the face or the regions that need to be stored separately so that it can be simply and quickly separated from the whole in the process of processing, and the topological relationship between them remains unchanged. This model needs to be able to clearly mark which points are feature points. In the process of access, it can quickly locate and access any vertex in the model anytime and anywhere. It can quickly find the edges associated with vertices.


### 3.3. Face Segmentation Method

Simply block the face, then train the corresponding CNN module for each block, and extract the feature cascade of the full connection layer, which can improve the accuracy and efficiency of face recognition. The bone block of the face is used as a scaffold, which is a framework to support facial muscles and skin. When the expression occurs, each bone block moves, regardless of the thickness of the bone block, and the bone block will not undergo elastic deformation. When the movement occurs, there is only geometric deformation. At the same time, considering the convenience of controlling the face model, it is assumed that the topological structure of facial bone blocks is also consistent. As an important part of facial movement, muscle is a layer of soft tissue with a certain thickness attached to the bone frame. Skin is used as an elastic mold to wrap stock price and muscle, and its thickness is different everywhere; And there are many fascia connecting facial structures between skeleton, muscle, and skin, which also have a certain impact, and the thickness is also different.For the face model, in order to facilitate research and control, the whole face is divided into blocks according to the distribution of organs and muscles of the face. It is divided into forehead, left and right parts of eyebrows, left and right parts of eyes, left and right parts of temples, and left and right parts of cheeks, followed by nose, mouth, and chin, as well as other transition parts. According to this distribution, a 3D face geometric model based on facial structure division is formed.


### 3.4. Global Feature Extraction of Facial Expression

The extraction of facial expression features plays an important role in the follow-up animation expression control. In the whole animation expression control, the extraction effect of facial expression features is the primary character. Firstly, the facial expression is holistic, and various parts and organs cooperate to complete the corresponding expression, although the degree of participation of some organs is less than that of other organs. However, when expression occurs, each region is more or less involved. Secondly, when expressing emotions, each different person is affected by different environment, culture, personality, and personal experience. Therefore, when expressing emotions, their forms of expression are diverse and different. In the production process, in addition to the predictable results in a specific environment, it can also be carried out in some cases, such as when the environment changes and becomes complex, so it also has a certain anti-interference ability. Finally, the extraction process should also achieve due speed and efficiency.Everyone is an independent individual, and the cultures of each country and region are very different. For example, the Chinese people are delicate and introverted in expressing their emotions, while the European and American people are open and exaggerated in expressing their emotions. Therefore, it is difficult to recognize the facial organ characteristics of each expression in the extraction process. Through experiments, researchers found that the universality of expression forms limited the expression to seven modes, reducing the difficulty of expression recognition.In the selection of feature points, if only all regions contained in the whole face are selected, such as nose, eyes, eyebrows, and other regions, the dimension of feature extraction will be increased, and the corresponding operation time will be increased, which cannot achieve the ideal effect. And some extracted feature points cannot express the motion characteristics and laws of the region they represent. Often these feature points will be useless in postproduction and become huge redundant data. This has an impact on identification production.Therefore, in order to change this disadvantage, researchers extract feature points through the global features of the face, in order to quickly distinguish the types with large differences in each expression. The most intuitive influence of the expression made by the face in expressing emotion is the changes in eyes, eyebrows, nose, and mouth. In these organs, the mouth area, eye area, and eyebrow area are more obvious, especially the mouth. Therefore, these parts play an important role in facial expression. In the extraction of feature points, we should focus on these parts, the most representative key points of these parts.


### 3.5. Extraction of Facial Feature Vector

Due to the great difference in human face, only relying on simple feature point extraction cannot meet more accurate calculation.Therefore, it is necessary to extract the feature vector in the calibrated feature region. In the whole facial expression, the mouth and eyebrow regions are more recognizable than other regions. Different weights are set for these different regions to lay a foundation for improving the recognition accuracy in the future.Gabor is a linear filter applied to edge extraction. Its filtering and direction expression are closer to the human visual system and the sensory system. It can provide good characteristics of direction selection and scale selection and is not sensitive to illumination changes. Therefore, it is widely used in the field of computer vision, so it can be applied to an expression.Gabor filter can be expressed as follows:(6)Jjx=∫Ixψjx−x2dx,whereJ (x) is the feature vector obtained by Gabor transformation of pixels x = (x, y) in facial expression image I (x) as a filter.ψjx is a Gabor kernel function used for feature extraction as follows:(7)Yjx=kj2s2exp−kj2x22s2expikjx−exp−s22,where kj is the wave vector parameter, including kjx and kjy two components.(8)kj=kjxkjy=kvcosϕukvsinϕu,where kv is the sampling frequency of the filter; v is the frequency coefficient, representing the wavelet transform scale; and ϕu is the direction coefficient, representing the direction of wavelet transform, and wavelet transforms the image in all directions and scales:(9)kv=2v+2/2πv=0,1,2,3,4,ϕu=uπ8u=0,1,2,3,4,5,6,7.The extraction range of feature vector should include mouth region, eye and eyebrow region, nose, and stria region. In order to facilitate the subsequent process, the corresponding weight is set for each region feature vector because each region changes differently when the expression occurs. Set all the extracted pixel points asp; set each part with dotted pixel points as pe,pm,pn; and set their respective weights as we,wm,wn.(10)we0=1,wm0=wn0=0.The feature vector of the image is extracted through the dimension of5∗8 and then multiplied by the weight coefficient to reach the initial threshold setting. At this time, the weight value is wet,wmt,wnt as follows:(11)we=wetwetpe+wmtpm+wntpn.Then, roughly locate the feature points of the whole face image, carry out convenient operations globally, and search the feature points of each region:(12)SJ,J′=∑n=1Nan′∑n=1Nan2∑n=1Nan′2.d⟶ is the position deviation between J,J′,ϕn,ϕn′, which indicates the phase response of the n the Eigenvector.In order to make the face matching recognition process more efficient and accurate, two algorithms are improved based on the original algorithm. One of the algorithms is called improved algorithm 1. Firstly, the face model is defined, and the feature points are marked. Roughly locate several basic parts on the image face and then calculate the feature points of each area to find out the law and determine the best position. Using the weighted feature vector, the image degree of the basic expression is calculated. The algorithm directly searches the segmented area, eliminates the search in the face range of the original algorithm, and reduces the search difficulty and time. In addition to this algorithm, there is another algorithm called improved algorithm 2. The previous steps are basically the same, but after rough positioning, the Euclidean distance between each feature point and the positioning face template will be calculated:(13)SJ,J′=∑n=1Nx−x′2+y−y′2′.More strictly match the eyes, eyebrows, and mouth areas. After the best position is found, the weight of the feature vector is used.Calculate the acquaintance degree between the basic expression and the positioning template as follows:(14)SJ,J′=J,J′JJ′,where J,J′ means to find the inner product and JJ′ is to operate the two ball binomials.Compared with the original algorithm, the two algorithms are simpler and have less computation.


### 3.6. Facial Eye Movement

Eye limits and coordinate system are shown in Figure5.Figure 5
Eye limits and coordinate system.Firstly, the displacement component of the upper eyelid is considered. Let the displacement component of any point of the upper eyelid displacement beU and V variables. According to the displacement variational method of elastic deformation, the formula is(15)Ux,y=u0x,y+∑mAmumx,y,Vx,y=v0x,y+∑mBmvmx,y,where umx,y and vmx,y are undetermined coefficients and u0x,y and v0x,y are displacement functions at the boundary satisfying eye boundary conditions, in which Am and Bm are also undetermined coefficients and independent of each other.According to the principle of elasticity, the displacement function of a certain point of upper eyelid displacement is as follows:(16)Ux,y=35cxy42b+20a1−x2a21−yb,Ux,y=1−x2a21−ybc+5cby16a2+2b2.The movement process of the eye moves along the horizontal and vertical directions, so the new coordinate of each vertex after deformation is(17)x′,y′,z′T=x,y,zT+u,v,0.The movement of our entire upper eyelid and lower eyelid is not infinitely extended, and there are certain restrictions on the movement radian, so we should also meet certain agreed conditions. In addition, the eyelid never leaves the eye surface during the whole movement process, so this movement process should also meet the spherical equation:(18)x−x02+y−y02+z−z02=r2.The coordinates of the eye centerx0,y0,z0 are the eye radius.By changing thec value to change the coordinate change, we can get the formula of the upper eyelid motion law: similarly, the motion principle of the lower eyelid is similar to that of the upper eyelid, but the lower eyelid is much smaller than that of the upper eyelid.(19)u0x,y=0,v0x,y=−d1−x2a2.Therefore, the solution and calculation method of the lower eyelid is the same as that of the eyelid, that is, the motion displacement equation of the lower eyelid is:(20)Ux,y=35axy42e+20axyae1−x2a21−yb,Ux,y=−d1−x2a2−5dey16a2+2e21−x2a21−yb.Similarly, the motion coordinates of the lower eyelid can be realized by changing thed value.


### 3.7. Closure of the Mouth of the Face

Similarly, the movement mode of the mouth is the same as that of the eyes. The mouth is regarded as an ellipse, which is reflected in the two-dimensional rectangular coordinate system. The closing of the mouth is in equilibrium, and the yawning is the maximum opening of the mouth. The movement of the mouth is regarded as the movement of an elastomer as shown in Figure6.Figure 6
Mouth boundary and coordinate system diagram.


## 3.1. Facial Features

There are about 7.8 billion people in the world. Everyone’s face looks different from others and has their own unique facial features. It is precise because people can distinguish different faces by relying on different facial features. Although each face is different and has its own characteristics, there are many kinds of the same movement rules in the face. For example, the expression of the face is the same, and the movement rules of each area of the face and internal muscle groups are the same; For example, the positional relationship of facial regions or organs will not change. According to the basic features of the face, the aforementioned “three courtyards and five eyes” are shown in Figure4.Figure 4
Overview of facial features.


## 3.2. Facial Region Segmentation

In the research of facial region segmentation, the initial researchers only regard the face as an overall structure to establish an animated character model, not specific to a certain region of the face. If you want to realize the expression change of a certain part or an organ or deal with the eyes, nose, or mouth separately, you will find that this implementation is very difficult. Staff requires advanced and complex technical requirements. Therefore, at first, the application of this model has certain limitations, and it cannot achieve the ideal effect. Later, in order to solve this problem, researchers proposed a 3D face machine model based on face segmentation. The biggest feature of this model is that it no longer treats the face as a whole but divides the whole face into multiple block structures according to a certain law. Each block structure is modeled and processed separately. Each block structure is independent, interdependent, and inseparable, so as to realize the structural mode of low cohesion and high coupling. Finally, these individually processed block structures can be combined again to form complete and different facial expressions. Although, on the surface, facial expressions look different, the face has many of the same properties, such as the basic proportional relationship of the three courtyards and five eyes of the face and the positional structure relationship of the face nose, eyes, and mouth, which will not change due to the difference of the face or the change of expression. Therefore, the researchers found that this provides the possibility for facial region segmentation or can be used as the basis of segmentation. The advantage of this model is that the face segmentation will not destroy the integrity of the face and the relative position and proportion relationship between various parts or organs and will not change due to the segmentation. In the process of modifying from a general face geometric model to a specific face geometric model, the face surface will not be seriously deformed. The animation provides great convenience for the production of facial expressions.At the same time, the researchers have shown in the experiment that the face is an elastomer. If the model is not adapted at the beginning and if a point on the model is moved arbitrarily during the interactive modification of the ordinary facial geometric model, it is necessary to calculate the position change of the vertices of the whole facial model. The amount of calculation required by this method is huge and complex, and in practical application, it wastes humans, materials, and resources. If the facial expression is divided into several small blocks, it is stipulated at the beginning that the motion of each point only affects other points in the block where the point is located, not the points of other blocks. In this way, it will simplify the complex and huge amount of calculation, significantly reduce the amount of calculation, and improve the accuracy and efficiency of calculation. Therefore, it is very necessary and effective to divide the face into regions.After the face is divided into blocks, in order to operate and process the geometric model of the face, it is necessary to design a good data structure for this geometric model. This data structure needs to meet the following requirements: there is a separate data block; this data block can be stored separately for the segmented organs or regions of the face or the regions that need to be stored separately so that it can be simply and quickly separated from the whole in the process of processing, and the topological relationship between them remains unchanged. This model needs to be able to clearly mark which points are feature points. In the process of access, it can quickly locate and access any vertex in the model anytime and anywhere. It can quickly find the edges associated with vertices.


## 3.3. Face Segmentation Method

Simply block the face, then train the corresponding CNN module for each block, and extract the feature cascade of the full connection layer, which can improve the accuracy and efficiency of face recognition. The bone block of the face is used as a scaffold, which is a framework to support facial muscles and skin. When the expression occurs, each bone block moves, regardless of the thickness of the bone block, and the bone block will not undergo elastic deformation. When the movement occurs, there is only geometric deformation. At the same time, considering the convenience of controlling the face model, it is assumed that the topological structure of facial bone blocks is also consistent. As an important part of facial movement, muscle is a layer of soft tissue with a certain thickness attached to the bone frame. Skin is used as an elastic mold to wrap stock price and muscle, and its thickness is different everywhere; And there are many fascia connecting facial structures between skeleton, muscle, and skin, which also have a certain impact, and the thickness is also different.For the face model, in order to facilitate research and control, the whole face is divided into blocks according to the distribution of organs and muscles of the face. It is divided into forehead, left and right parts of eyebrows, left and right parts of eyes, left and right parts of temples, and left and right parts of cheeks, followed by nose, mouth, and chin, as well as other transition parts. According to this distribution, a 3D face geometric model based on facial structure division is formed.


## 3.4. Global Feature Extraction of Facial Expression

The extraction of facial expression features plays an important role in the follow-up animation expression control. In the whole animation expression control, the extraction effect of facial expression features is the primary character. Firstly, the facial expression is holistic, and various parts and organs cooperate to complete the corresponding expression, although the degree of participation of some organs is less than that of other organs. However, when expression occurs, each region is more or less involved. Secondly, when expressing emotions, each different person is affected by different environment, culture, personality, and personal experience. Therefore, when expressing emotions, their forms of expression are diverse and different. In the production process, in addition to the predictable results in a specific environment, it can also be carried out in some cases, such as when the environment changes and becomes complex, so it also has a certain anti-interference ability. Finally, the extraction process should also achieve due speed and efficiency.Everyone is an independent individual, and the cultures of each country and region are very different. For example, the Chinese people are delicate and introverted in expressing their emotions, while the European and American people are open and exaggerated in expressing their emotions. Therefore, it is difficult to recognize the facial organ characteristics of each expression in the extraction process. Through experiments, researchers found that the universality of expression forms limited the expression to seven modes, reducing the difficulty of expression recognition.In the selection of feature points, if only all regions contained in the whole face are selected, such as nose, eyes, eyebrows, and other regions, the dimension of feature extraction will be increased, and the corresponding operation time will be increased, which cannot achieve the ideal effect. And some extracted feature points cannot express the motion characteristics and laws of the region they represent. Often these feature points will be useless in postproduction and become huge redundant data. This has an impact on identification production.Therefore, in order to change this disadvantage, researchers extract feature points through the global features of the face, in order to quickly distinguish the types with large differences in each expression. The most intuitive influence of the expression made by the face in expressing emotion is the changes in eyes, eyebrows, nose, and mouth. In these organs, the mouth area, eye area, and eyebrow area are more obvious, especially the mouth. Therefore, these parts play an important role in facial expression. In the extraction of feature points, we should focus on these parts, the most representative key points of these parts.


## 3.5. Extraction of Facial Feature Vector

Due to the great difference in human face, only relying on simple feature point extraction cannot meet more accurate calculation.Therefore, it is necessary to extract the feature vector in the calibrated feature region. In the whole facial expression, the mouth and eyebrow regions are more recognizable than other regions. Different weights are set for these different regions to lay a foundation for improving the recognition accuracy in the future.Gabor is a linear filter applied to edge extraction. Its filtering and direction expression are closer to the human visual system and the sensory system. It can provide good characteristics of direction selection and scale selection and is not sensitive to illumination changes. Therefore, it is widely used in the field of computer vision, so it can be applied to an expression.Gabor filter can be expressed as follows:(6)Jjx=∫Ixψjx−x2dx,whereJ (x) is the feature vector obtained by Gabor transformation of pixels x = (x, y) in facial expression image I (x) as a filter.ψjx is a Gabor kernel function used for feature extraction as follows:(7)Yjx=kj2s2exp−kj2x22s2expikjx−exp−s22,where kj is the wave vector parameter, including kjx and kjy two components.(8)kj=kjxkjy=kvcosϕukvsinϕu,where kv is the sampling frequency of the filter; v is the frequency coefficient, representing the wavelet transform scale; and ϕu is the direction coefficient, representing the direction of wavelet transform, and wavelet transforms the image in all directions and scales:(9)kv=2v+2/2πv=0,1,2,3,4,ϕu=uπ8u=0,1,2,3,4,5,6,7.The extraction range of feature vector should include mouth region, eye and eyebrow region, nose, and stria region. In order to facilitate the subsequent process, the corresponding weight is set for each region feature vector because each region changes differently when the expression occurs. Set all the extracted pixel points asp; set each part with dotted pixel points as pe,pm,pn; and set their respective weights as we,wm,wn.(10)we0=1,wm0=wn0=0.The feature vector of the image is extracted through the dimension of5∗8 and then multiplied by the weight coefficient to reach the initial threshold setting. At this time, the weight value is wet,wmt,wnt as follows:(11)we=wetwetpe+wmtpm+wntpn.Then, roughly locate the feature points of the whole face image, carry out convenient operations globally, and search the feature points of each region:(12)SJ,J′=∑n=1Nan′∑n=1Nan2∑n=1Nan′2.d⟶ is the position deviation between J,J′,ϕn,ϕn′, which indicates the phase response of the n the Eigenvector.In order to make the face matching recognition process more efficient and accurate, two algorithms are improved based on the original algorithm. One of the algorithms is called improved algorithm 1. Firstly, the face model is defined, and the feature points are marked. Roughly locate several basic parts on the image face and then calculate the feature points of each area to find out the law and determine the best position. Using the weighted feature vector, the image degree of the basic expression is calculated. The algorithm directly searches the segmented area, eliminates the search in the face range of the original algorithm, and reduces the search difficulty and time. In addition to this algorithm, there is another algorithm called improved algorithm 2. The previous steps are basically the same, but after rough positioning, the Euclidean distance between each feature point and the positioning face template will be calculated:(13)SJ,J′=∑n=1Nx−x′2+y−y′2′.More strictly match the eyes, eyebrows, and mouth areas. After the best position is found, the weight of the feature vector is used.Calculate the acquaintance degree between the basic expression and the positioning template as follows:(14)SJ,J′=J,J′JJ′,where J,J′ means to find the inner product and JJ′ is to operate the two ball binomials.Compared with the original algorithm, the two algorithms are simpler and have less computation.


## 3.6. Facial Eye Movement

Eye limits and coordinate system are shown in Figure5.Figure 5
Eye limits and coordinate system.Firstly, the displacement component of the upper eyelid is considered. Let the displacement component of any point of the upper eyelid displacement beU and V variables. According to the displacement variational method of elastic deformation, the formula is(15)Ux,y=u0x,y+∑mAmumx,y,Vx,y=v0x,y+∑mBmvmx,y,where umx,y and vmx,y are undetermined coefficients and u0x,y and v0x,y are displacement functions at the boundary satisfying eye boundary conditions, in which Am and Bm are also undetermined coefficients and independent of each other.According to the principle of elasticity, the displacement function of a certain point of upper eyelid displacement is as follows:(16)Ux,y=35cxy42b+20a1−x2a21−yb,Ux,y=1−x2a21−ybc+5cby16a2+2b2.The movement process of the eye moves along the horizontal and vertical directions, so the new coordinate of each vertex after deformation is(17)x′,y′,z′T=x,y,zT+u,v,0.The movement of our entire upper eyelid and lower eyelid is not infinitely extended, and there are certain restrictions on the movement radian, so we should also meet certain agreed conditions. In addition, the eyelid never leaves the eye surface during the whole movement process, so this movement process should also meet the spherical equation:(18)x−x02+y−y02+z−z02=r2.The coordinates of the eye centerx0,y0,z0 are the eye radius.By changing thec value to change the coordinate change, we can get the formula of the upper eyelid motion law: similarly, the motion principle of the lower eyelid is similar to that of the upper eyelid, but the lower eyelid is much smaller than that of the upper eyelid.(19)u0x,y=0,v0x,y=−d1−x2a2.Therefore, the solution and calculation method of the lower eyelid is the same as that of the eyelid, that is, the motion displacement equation of the lower eyelid is:(20)Ux,y=35axy42e+20axyae1−x2a21−yb,Ux,y=−d1−x2a2−5dey16a2+2e21−x2a21−yb.Similarly, the motion coordinates of the lower eyelid can be realized by changing thed value.


## 3.7. Closure of the Mouth of the Face

Similarly, the movement mode of the mouth is the same as that of the eyes. The mouth is regarded as an ellipse, which is reflected in the two-dimensional rectangular coordinate system. The closing of the mouth is in equilibrium, and the yawning is the maximum opening of the mouth. The movement of the mouth is regarded as the movement of an elastomer as shown in Figure6.Figure 6
Mouth boundary and coordinate system diagram.


## 4. Experimental Comparison and Analysis

The improved algorithms of the above four methods are compared by experiments. Experiments verify the operability and effectiveness of the algorithm. This experiment mainly uses the Jaffe database and the Cohn-Kanade database.There are 213 images in the Jaffe data set. They were divided into 7 expression types, namely, anger, disgust, fear, happiness, sadness, surprise, and neutrality. Each expression is different in degree as shown in Figure7.Figure 7
Schematic diagram of Jaffe database sample.The Cohn-Kanade database includes 593 expression sequences of the faces of 123 male and female individuals with different skin colors and different ages. Some individuals have 7 expression sequences, and the rest have 6 expression sequences other than disgusting expressions. The expression sequence is from calm to peak expression. FACS was used to encode the expression peak as shown in Figure8.Figure 8
Schematic diagram of Cohn-Kanade database sample.In the experiment, 213 images were selected from the Jaffe library, and 97 individuals were selected from the Cohn-Kanade library, including 692 images. The so-calledk-fold cross-validation means that in each experiment, the samples are evenly divided into k parts, and the training samples and the samples to be tested are evenly divided according to a certain proportion, which is usually distributed according to the proportion of k − 1:1. After K experiments are repeated, the results of each experiment are averaged to obtain the final results of the experiment. In the experiment done in this paper, the 50% cross-validation is selected, but because the number of database samples cannot meet the absolute bisection, the sample data is approximately divided into 5 bisections: 3 are training samples and 2 are test samples. In the experiment, all pictures were changed in the size of 90∗100, and the 15∗4 area of the eye eyebrow area and the 8∗6 area of the mouth area were selected for feature extraction. The total eye eyebrow area was given a weight value of 0.46; the mouth area was given a weight value of 0.36; and the last 0.18 was given to the nose area. In the experiment, for the sake of fairness, the positioning face data used are not selected from the experimental data.


### 4.1. Comparison of Experimental Results

The recognition results of the original algorithm and the improved algorithm in the Cohn-Kanade library are shown in the Figure9. The recognition results are obtained from the Jaffe library. The abscissa is the parameter of the Gabor transform, and the ordinate is the average recognition rate. When the feature extraction dimension is too small, the feature is not enough to express more complete information. Therefore, the experiment is carried out when the Gabor scale parameters, and direction parameters are greater than half of the full extraction (8∗5) as shown in Figures 9 and 10.Figure 9
Comparison of recognition results of three algorithms in the CK database.Figure 10
Comparison of recognition results of three algorithms in the Jaffe Library.From the experimental results and the recognition rate, the improved algorithms 1 and 2 are much better than the original algorithm. The expressions in the Cohn-Kanade library are the facial expressions of European and American characters, and their expressions are put out. The expression of Japanese female characters in the Jaffe library is more implicit than that of Europeans and Americans. Therefore, the expression recognition difficulty in the Cohn-Kanade library is slightly less than that in the Jaffe library, and the recognition rate will be slightly higher than that in the Jaffe library.Average one face image processed by the three algorithms, in milliseconds, as shown in Figure11.Figure 11
Comparison of average processing time of one image by three algorithms.Through comparative research, it is found that as shown in Figure11, the efficiency of the improved two algorithms is significantly higher than that of the original algorithm. Therefore, it is necessary to study the algorithm. Especially in algorithm 2, the operation time is significantly reduced.In order to accurately identify each expression type, separate data statistics are mainly carried out for algorithm 2. The recognition rate of happy and surprised expressions in the Cohn-Kanade database is the highest, reaching more than 95%; the recognition rate of anger, disgust, sadness, and neutrality is close, reaching more than 85%; and the recognition rate of fear is the lowest, only 83.7%. On the other hand, the confusion probability of anger, disgust, and fear is the highest. These three expressions are also relatively difficult to identify. They all have a high probability of being misjudged as the other two expressions, and the probability of fear being misjudged as a surprise is the highest, up to 8.2%. In the Jaffe library, the highest recognition rate is still happy and surprised, which are more than 93%, and the lowest is the recognition accuracy of neutral expressions, which is less than 80%, and the recognition rates of other types are more than 80%. In the Jaffe library, the confusion probability between expressions is higher than that in the Cohn-Kanade library, especially the errors misjudged as disgusting and neutral are the most common. The confusion probability among anger, disgust, and neutral is the most obvious. Anger, neutral, and fear all have a probability of nearly 8% to be wrongly identified as disgust. The confusion matrix in the Cohn-Kanade library and the Jaffe library is shown in Tables2 and 3.Table 2
Confusion matrix of algorithm 3 in the CK database.
AngerHateFearHappySadSurprisedNeutralAnger85.65.73.80.04.90.00.0Hate1.391.31.70.00.54.30.9Fear3.83.483.70.50.38.20.1Happy0.20.00.797.80.40.90.0Sad1.92.72.20.586.80.85.1Surprised0.10.70.53.30.195.30.0Neutral3.61.42.90.53.11.886.7Table 3
Confusion matrix of algorithm 3 in the Jaffe Library.
AngerHateFearHappySadSurprisedNeutralAnger83.18.02.70.02.30.03.9Hate4.285.85.50.21.70.02.6Fear2.07.981.12.40.53.3.2.8Happy0.50.01.096.10.32.10.0Sad1.80.93.20.190.40.03.6Surprised0.51.70.42.90.093.70.8Neutral5.98.50.82.70.83.477.9By comprehensively comparing the above two figures, it is not difficult to find that the recognition rate of happiness and surprise is the highest in the two databases because the changes in these two kinds of expressions are the most obvious, while the recognition rate of fear expressions does not exceed 85% on both sides, which may be misjudged as surprise and disgust. In essence, the pupils of surprise are enlarged; the mouth is open; and the frown of disgust. The changes in facial organs such as open mouth coincide with the changes of fear expression, so it has a high probability of misjudgment. In particular, the recognition rate of neutral expression on Jaffe database is lower than 80%. The reason is that the expression on the Jaffe database is more implicit, the change of facial features is relatively not obvious, and it is difficult to locate and extract feature points. Therefore, it has a high probability of mutual misjudgment with another disgusting expression with little change of facial features.In order to compare the methods used in this paper with other similar methods, this paper counts the relevant global feature recognition effects and applies the methods of FACS algorithm, PCA algorithm, and feature vector + texture information because, in some papers, only six kinds of expression recognition except neutral expression are used. In addition, some of the methods used are not tested in the same database and are not comparable. What this paper does is the recognition of seven expressions. Therefore, some open-source algorithms are tested, and the other algorithms are counted according to the experimental data of relevant papers. Comparing the recognition rate in the Cohn-Kanade library, this method refers to algorithm 2, which performs feature extraction under the condition of6∗3. Jaffe database is the same as shown in Figures 12 and 13.Figure 12
Comparison of recognition rate of global feature database.Figure 13
Comparison of recognition rate of global feature Jaffe library.In the Cohn-Kanade library, the overall recognition rate of this method is improved compared with the relevant global feature methods, and the recognition rate of some types, such as happy and surprised, is improved, but the recognition method of neutral and angry expressions is slightly inferior. The reason is that these kinds of expressions, especially neutral expressions, pay more attention to the distinction between details. Neutral expressions and expressions with little change in other facial organs, such as disgust and anger, are less distinguished from each other, so it is difficult to distinguish them as a whole. Compared with the vector texture algorithm, the feature point topology of facial organs focuses more on the overall characteristics of the whole world. Therefore, the recognition results of neutral and angry classes are not as good as the texture method.The comparison of the recognition rate on the Jaffe database shows that the overall recognition rate of this method on the Jaffe database is improved compared with the relevant global feature methods, but the judgment of some expressions, such as disgusting expressions, needs to be improved. The reason is to improve the speed of recognition, so as to reduce the feature points that contribute less to recognition, but at the same time, their role is completely reduced. Therefore, for some expression types that need more feature points to be accurately defined, the results obtained are not as good as those using more feature points, and the definition of aversion class is also more difficult, Therefore, the recognition rate of this kind is reduced.After the whole experiment, through the experimental data, it can be seen that the extraction method of the facial feature vector in face division is very important for animation expression control. This extraction method can reduce the amount of data calculation, improve efficiency, and remove redundancy.


### 4.2. Experimental Summary

Through the comparison of several algorithms in the above experiments, it is not difficult to see that the improved algorithm is obviously superior to the original algorithm in terms of computational efficiency and accuracy. This algorithm can achieve better results under the Cohn-Kanade library and the Jaffe library. Therefore, in the division of facial features and facial regions, it is very important to extract facial feature vectors. This extraction method can reduce the amount of data calculation, improve efficiency, and remove redundancy. It can lay a good foundation for subsequent animation expression control.


## 4.1. Comparison of Experimental Results

The recognition results of the original algorithm and the improved algorithm in the Cohn-Kanade library are shown in the Figure9. The recognition results are obtained from the Jaffe library. The abscissa is the parameter of the Gabor transform, and the ordinate is the average recognition rate. When the feature extraction dimension is too small, the feature is not enough to express more complete information. Therefore, the experiment is carried out when the Gabor scale parameters, and direction parameters are greater than half of the full extraction (8∗5) as shown in Figures 9 and 10.Figure 9
Comparison of recognition results of three algorithms in the CK database.Figure 10
Comparison of recognition results of three algorithms in the Jaffe Library.From the experimental results and the recognition rate, the improved algorithms 1 and 2 are much better than the original algorithm. The expressions in the Cohn-Kanade library are the facial expressions of European and American characters, and their expressions are put out. The expression of Japanese female characters in the Jaffe library is more implicit than that of Europeans and Americans. Therefore, the expression recognition difficulty in the Cohn-Kanade library is slightly less than that in the Jaffe library, and the recognition rate will be slightly higher than that in the Jaffe library.Average one face image processed by the three algorithms, in milliseconds, as shown in Figure11.Figure 11
Comparison of average processing time of one image by three algorithms.Through comparative research, it is found that as shown in Figure11, the efficiency of the improved two algorithms is significantly higher than that of the original algorithm. Therefore, it is necessary to study the algorithm. Especially in algorithm 2, the operation time is significantly reduced.In order to accurately identify each expression type, separate data statistics are mainly carried out for algorithm 2. The recognition rate of happy and surprised expressions in the Cohn-Kanade database is the highest, reaching more than 95%; the recognition rate of anger, disgust, sadness, and neutrality is close, reaching more than 85%; and the recognition rate of fear is the lowest, only 83.7%. On the other hand, the confusion probability of anger, disgust, and fear is the highest. These three expressions are also relatively difficult to identify. They all have a high probability of being misjudged as the other two expressions, and the probability of fear being misjudged as a surprise is the highest, up to 8.2%. In the Jaffe library, the highest recognition rate is still happy and surprised, which are more than 93%, and the lowest is the recognition accuracy of neutral expressions, which is less than 80%, and the recognition rates of other types are more than 80%. In the Jaffe library, the confusion probability between expressions is higher than that in the Cohn-Kanade library, especially the errors misjudged as disgusting and neutral are the most common. The confusion probability among anger, disgust, and neutral is the most obvious. Anger, neutral, and fear all have a probability of nearly 8% to be wrongly identified as disgust. The confusion matrix in the Cohn-Kanade library and the Jaffe library is shown in Tables2 and 3.Table 2
Confusion matrix of algorithm 3 in the CK database.
AngerHateFearHappySadSurprisedNeutralAnger85.65.73.80.04.90.00.0Hate1.391.31.70.00.54.30.9Fear3.83.483.70.50.38.20.1Happy0.20.00.797.80.40.90.0Sad1.92.72.20.586.80.85.1Surprised0.10.70.53.30.195.30.0Neutral3.61.42.90.53.11.886.7Table 3
Confusion matrix of algorithm 3 in the Jaffe Library.
AngerHateFearHappySadSurprisedNeutralAnger83.18.02.70.02.30.03.9Hate4.285.85.50.21.70.02.6Fear2.07.981.12.40.53.3.2.8Happy0.50.01.096.10.32.10.0Sad1.80.93.20.190.40.03.6Surprised0.51.70.42.90.093.70.8Neutral5.98.50.82.70.83.477.9By comprehensively comparing the above two figures, it is not difficult to find that the recognition rate of happiness and surprise is the highest in the two databases because the changes in these two kinds of expressions are the most obvious, while the recognition rate of fear expressions does not exceed 85% on both sides, which may be misjudged as surprise and disgust. In essence, the pupils of surprise are enlarged; the mouth is open; and the frown of disgust. The changes in facial organs such as open mouth coincide with the changes of fear expression, so it has a high probability of misjudgment. In particular, the recognition rate of neutral expression on Jaffe database is lower than 80%. The reason is that the expression on the Jaffe database is more implicit, the change of facial features is relatively not obvious, and it is difficult to locate and extract feature points. Therefore, it has a high probability of mutual misjudgment with another disgusting expression with little change of facial features.In order to compare the methods used in this paper with other similar methods, this paper counts the relevant global feature recognition effects and applies the methods of FACS algorithm, PCA algorithm, and feature vector + texture information because, in some papers, only six kinds of expression recognition except neutral expression are used. In addition, some of the methods used are not tested in the same database and are not comparable. What this paper does is the recognition of seven expressions. Therefore, some open-source algorithms are tested, and the other algorithms are counted according to the experimental data of relevant papers. Comparing the recognition rate in the Cohn-Kanade library, this method refers to algorithm 2, which performs feature extraction under the condition of6∗3. Jaffe database is the same as shown in Figures 12 and 13.Figure 12
Comparison of recognition rate of global feature database.Figure 13
Comparison of recognition rate of global feature Jaffe library.In the Cohn-Kanade library, the overall recognition rate of this method is improved compared with the relevant global feature methods, and the recognition rate of some types, such as happy and surprised, is improved, but the recognition method of neutral and angry expressions is slightly inferior. The reason is that these kinds of expressions, especially neutral expressions, pay more attention to the distinction between details. Neutral expressions and expressions with little change in other facial organs, such as disgust and anger, are less distinguished from each other, so it is difficult to distinguish them as a whole. Compared with the vector texture algorithm, the feature point topology of facial organs focuses more on the overall characteristics of the whole world. Therefore, the recognition results of neutral and angry classes are not as good as the texture method.The comparison of the recognition rate on the Jaffe database shows that the overall recognition rate of this method on the Jaffe database is improved compared with the relevant global feature methods, but the judgment of some expressions, such as disgusting expressions, needs to be improved. The reason is to improve the speed of recognition, so as to reduce the feature points that contribute less to recognition, but at the same time, their role is completely reduced. Therefore, for some expression types that need more feature points to be accurately defined, the results obtained are not as good as those using more feature points, and the definition of aversion class is also more difficult, Therefore, the recognition rate of this kind is reduced.After the whole experiment, through the experimental data, it can be seen that the extraction method of the facial feature vector in face division is very important for animation expression control. This extraction method can reduce the amount of data calculation, improve efficiency, and remove redundancy.


## 4.2. Experimental Summary

Through the comparison of several algorithms in the above experiments, it is not difficult to see that the improved algorithm is obviously superior to the original algorithm in terms of computational efficiency and accuracy. This algorithm can achieve better results under the Cohn-Kanade library and the Jaffe library. Therefore, in the division of facial features and facial regions, it is very important to extract facial feature vectors. This extraction method can reduce the amount of data calculation, improve efficiency, and remove redundancy. It can lay a good foundation for subsequent animation expression control.


## 5. Conclusion

Nowadays, various animation production technologies continue to emerge. These animation technologies have a super exquisite and realistic appearance and bring a strong visual impact to the audience. Animation expression control technology is an essential branch of animation technology. Technicians are required to be able to produce an animated expression similar to the structural image of facial expression, which requires technicians to understand its internal structure and be very familiar with it. This is not only related to the mastery of animation technology but also related to psychology, anatomy, and other aspects. The division of facial expression structure divides the complex expression structure of the face and simplifies the complex problem, so as to solve the difficult problem of animation control technology. Therefore, the division of facial modules is essential for animation expression control technology. With the gradual globalization of the animation industry, it has penetrated people’s lives and becomes an indispensable part of life. Relevant technicians should strengthen their research. For animation production, character animation plays a leading role, and the research of character face control technology is very important for animation production. It may become the main research and development direction of technicians in the future. Only by constantly pushing through the old and bringing forth the new and constantly innovating facial expression control technology can we promote the development of the animation industry.



---

*Source: 5800099-2022-05-12.xml*