# Construction of Facial Expression Recognition Technology of Intelligent Service Robot Based on Nanofiber Flexible Wearable Electronic Skins

Authors: Feigao Li; Chenyang Li; Yue Liu
Journal: Advances in Materials Science and Engineering
 (2022)

---

## Abstract

Recognizing human features is a highly significant and popular research topic in academia today, especially with regard to human intervention and computer intelligence. If you want your computer to have real intelligence, then the computer must first learn to understand human feelings, and facial expressions often contain rich emotional information. Clearly, recognizing human features is indispensable for the future intelligent development of computers. There are always a few components and many researchers are involved. Compared with the traditional facial expression recognition method, which needs to manually define features and artificially perform appropriate feature extraction, a great advantage of deep learning is to remove the disadvantages that require a lot of human intervention, thereby reducing the uncertainty of expression recognition. In recent years, deep processing has led to outstanding gains in numerous domains, including robotics, computer graphics, and natural language processing. Therefore, this article applies a popular recognition algorithm based on deep learning, combined with a nanofiber flexible wearable electronic skin, and applies it to the field of facial expression recognition. Functional expression identification is an essential goal of the external analysis of human features. Mature facial expression recognition technology has great application values.


---

## Body

## 1. Introduction

Smart service robots based on nanofiber flexible wearable electronic skins are various high-tech integrated smart devices that can provide humans with necessary services in an unstructured environment mainly focusing on the application requirements of service robots and dangerous work robots. Intelligent service robot technology based on nanofiber flexible wearable electronic skins involves many fields, such as machinery, electronics, materials, computers, sensors, control, and medicine. It requires brainstorming and multifaceted collaborative research [1]. The robot must be able to understand and respond to people’s needs by listening, speaking, observing people’s movements, expressions, etc. In an unstructured environment, robots must quickly collect and quickly process the necessary information for all environments, including a wide variety of static and dynamic objects, including humans and animals. Advanced hybrid drive technology, flexible robot technology, tactile technology, visual technology, and artificial intelligence technology make service robots, especially humanoid service robots, smarter, and more friendly. Not only can they walk, run, communicate and express emotions but they can also learn and grow their intelligence like children. The key technologies that China is currently mastering are, namely, environmentally aware sensors and signal processing methods. The service robot must observe and identify the surrounding environment while working. The more sensors, the more information the surrounding environment acquires and the more complementary and redundant the various sensor information is in space and time, the more they are combined by some kind of operation and optimization. To describe and accurately identify the environment, intelligent control is the key technology of the robot that mainly includes the following: neural network, paste control, and optimization calculation. Fuzzy control is a control method that uses the basic theory of fuzzy mathematics. It generally has the following 5 principal sections: defining parameters, ambiguities, a repository of information, illogical determination, and defuzzification. Artificial neural network control is inferred from the activities of humans and creatures [2]. The characteristics of neural networks are information processing and reasoning, association, and thinking. Navigation and positioning: The multisensor fusion technology and artificial neural network control in the service robot system can realize the autonomous positioning and navigation of the service robot. Route plan: Path planning refers to finding a route between the initial situation and the desired goal situation within the service motor’s construction space in order to escape from barriers. The artificial intelligence approach is a developed and productive programming approach to conventional arithmetic. It allows route programming processes using AI tools but without thinking about whether the route is best or not. Smart route programming approaches can provide greater barrier avoidance performance, accelerate programming faster, and enable real-world solutions. Human-robot interaction technology. Since the aim of smart computing research is to enable robots to work smartness and serve humans as if they had a “brain,” there must be human instruction and domination, detection, and maintenance of the robot, and feedback from the robot on the outcomes of its operations. It mainly includes technologies that include texture awareness, synthesizing and recording spoken language, identifying and managing pictures, and performing computer interpretation [3]. At present, the research on the more popular intelligent service robots in foreign countries is mainly concentrated in Germany, Japan, and the United States and has already been adopted by a variety of companies. Due to the economic prosperity, the increasing demand for life in China and the increase in the proportion of elderly people caused by family planning policies, and the problem of elderly care, many robotic R&D companies have begun to shift their research focus to the development of intelligent service robots and have made relatively large achievements in this area.Following the rapid growth of synthetic AI and computer vision, there is a growing demand for intelligent emotional feedback and anthropomorphic person-machine engagement. As one of the key components of AI human-computer communication technology, the recognition of functional facial expressions has attracted a lot of focus in recent years. Human expressions can be divided into macro expressions and microexpressions. The study of macroscopic expressions has been more than 20 years old and has achieved certain results. Microexpressions are spontaneous and unconscious expressions in a very short time. Changes, which occur for a short time, are difficult to capture by humans. Therefore, high-speed cameras need to be introduced to capture and analyze them one frame at a time. However, this work is still not something that ordinary people can do and requires some experience. The psychology expert can complete it. What we have to do is to help humans recognize expressions through deep learning. Facial expression recognition refers to the analysis of human facial expressions and changes through computer pattern recognition and machine learning algorithms and then judges people’s inner emotions and thought activities, thereby obtaining a more intelligent human-computer interaction environment. The machine learning method based on traditional manual custom feature classification has gradually become the main method for the automatic recognition of facial expressions. A series of research results have been obtained, but functional features are influenced by numerous parameters, such as the influence of the natural environment, such as illumination and angle. as Along with the influence of individual differences in ethnicity, gender, age, etc., traditional methods have many difficulties in manually customizing features [4]. The concepts and advancements of profound training have provided innovative ideas and approaches for researchers working in the visual identity field. It replaces artificial signature collection in the form of effective techniques for unmonitored or quasi-supervised signature acquisition and stratified signature capture. Fruitful results. Thanks to the use of the graphic processing unit (GPU) and the improvement of computing performance, the convolutional neural network (CNN) has achieved a series of eye-catching performances in the field of image recognition, including AlexNet and VGGNet. The network structure such as ResNet has won many times in the ILSVRC competition, and its results have also been applied to various image recognition tasks. Currently, profound training, specifically convergent cognitive approaches to facial expression recognition, has become the mainstream approach to automatic facial expression recognition. However, there are also difficult problems in the research of facial expression recognition, which are summarized as follows: The human face is a flexible body in a three-dimensional space. It is easy and accurate to build a rigid body model, but the human face is a flexible body in a three-dimensional space, and it is difficult to establish a model. The existence of individual differences affects expression recognition. The experimental results are different for the same expression, due to the influence of individual differences. If the individual difference is greater, the influence on the expression recognition will be greater. When feature extraction, the interference of individual differences cannot be completely avoided, which undoubtedly increases the difficulty of extracting expression features of the same category. The expression itself is different. The expressions of Europeans and Americans are more exaggerated, the emotions are more obvious, while the Asians are more restrained and the expressions are smaller; women and children express more expressive expressions than men; the expressions of old and young people are different. The folds also affect the extraction of expression features and expression recognition in the natural environment. At present, most expression recognition is performed on the published database, and the images in these databases are expressions of the unnatural environment made by volunteers under the guidance of professionals. Most of them are exaggerated and not true to volunteers. The expression of emotions has certain performance components, which are not conducive to the application of actual scenes. Interference from external factors. The existence of disturbance factors such as illumination, attitude change, and noise in the actual environment not only increases the difficulty of obtaining effective expression information but also is not conducive to popularizing the expression recognition system in practical applications.About the expression recognition since the beginning of the 21th century, based on a simple knowledge distillation scheme, Gera D uses a single network to complete label inference on large-scale facial expression datasets, and the proposed framework demonstrates its effectiveness and comprehensiveness in the noisy FER dataset [5]. Li et al. fused energy feature vectors and facial feature vectors in their research study, used support vector machine (SVM) to classify the fused feature vector, and finally proved that the proposed method has higher accuracy and stronger generalization transformation ability [6]. Hossain and Yousuf proposed a facial recognition system that uses the currently developed OpenCV Haar Feature-based cascade classifier for face detection, capable of recognizing natural head nods and shakes [7]. Jan and Sajjad proposed a new descriptor called the Weber discrete wavelet transform, which can effectively identify facial expressions, and finally, they observed through experiments that the feature dimension was significantly reduced, while the recognition accuracy was significantly improved [8]. Wang et al. proposed a multi-manifold discriminant analysis method for image feature extraction and finally verified the effectiveness of this analysis method for image feature extraction through experiments [9]. Rusia and Singh proposed two customized CNN models, namely, Proposed Model 1 and Proposed Model 2, to classify generic facial expressions to prevent overfitting [10]. These research methods have carried out a profound research on face recognition to a certain extent.The emergence of deep learning in recent years has brought new vitality to the field of computer vision [11]. In fact, it has made the research of behavior recognition, face recognition, and semantic segmentation into a new stage and achieved very significant results. The artificial neural network is a very complex nonlinear network [12]. It is composed of a large number of processing units and is an abstraction and modeling of the human brain or biological neural network. Since the artificial neural network was proposed, after decades of development, researchers still cannot interpret or completely imitate the working mechanism of the brain. For a while, the research study only stays in the shallow neural network. In recent years, the development of deep neural networks has brought about another phase of neural network development. Deep learning is also a kind of neural network. Deep learning refers to the way of using artificial neural networks to learn features by relying on powerful graphics computing power and the development of big data at present. The basic idea is that through convolutional neural networks (CNN) and other methods to carry out complex calculations of a large number of training datasets and update network parameters through backpropagation, finally enabling the trained model to better reflect the data characteristics [13]. In order to solve the problem of expression recognition in traditional methods, deep learning seems to be a good tool. However, deep learning needs to be trained with a higher number of databases in order to obtain representative features, otherwise, it will cause over-fitting, etc. Problems. Following the triadic categorization scheme in profound understanding techniques and operations, the profound understanding structures are broadly classified into the following three types: (1) Asocial or spontaneous deep systems: The profound Internet produces symptoms validly through Internet coding without employing object tags, so it is also known as a generative method. For example, the defined branch neural network (DBN), revolving network nature (RNN), and the like. (2) Deep forests with monitored teaching: Significantly, superintended training means that profound systems use labeling of object types, either explicitly or obliquely, to create distinctions for the purpose of modality selection, so they are also known as distinguishing depth networks, such as curvilinear neural nets (CNNs) and reciprocal neural nets (RNNs). (3) Hybrid deep forests: “Hybrid” refers to a conjunction of unsupervised and monitored training, such as the DBN-CNN hybrid model for pretraining deep convolutional neural networks using a generative DBN.According to the aforesaid diagnosis, this article suggests that the facial expression recognition operation based on profound acquisition improves the accuracy of expression recognition, the average accuracy rate is improved by 2%, and it is faster than the popular expression recognition algorithm. Moreover, through the combination of deep face-based face verification algorithms, different expressions of the same person can be obtained, which effectively helps to further analyze facial expressions.


## 2. Proposed Method

Expression recognition can be deeply studied through deep learning [14]. The profound acquisition is a division of methodological training. The purpose is to simulate the structure of the brain’s neural network by building a model. When processing data (images, voices, texts, etc.), it is a multilevel structure to learn features. Deep learning can be divided into deep concessional bridging systems, profound conviction systems, profound Boltzmann systems, and compounded autoencoders according to the model structure. The deep involutional nerve structure is a type of recurrent nerve structure designed to process multidimensional data [15].


### 2.1. The Overall Framework of Expression Recognition Based on Deep Convolutional Neural Networks

In this section, a dynamic expression identification system using profound involutional integer systems is designed [16]. The system consists of a computer camera and software function modules. The video image information is collected in real time by the camera in a real environment, and each frame of the video is separately calculated. In the software part, first, the Haar classifier is used for face detection. Second, the obtained face image is preprocessed to obtain the face gray image; the obtained gray image is input to the trained deep convolutional neural network again. Finally, the resulting expression features are entered into the SoftMax classifier for expression classification. Real-time dynamic facial expression recognition [17].The block diagram of the expression recognition system based on deep convolutional neural network is shown in Figure1.Figure 1
Block diagram of the expression recognition system based on the deep convolutional neural network.


### 2.2. Face Detection Using the Haar Classifier

This section uses the face detection method based on the Haar-like feature and the AdaBoost algorithm, namely, the Haar classifier algorithm [18]. This algorithm uses the Viola–Jones target detection framework, which is a real-time object detection method, which can realize the first frame from video. It positions the face to achieve real-time expression recognition. The integral graph algorithm can be used to quickly calculate Haar-like features with different sizes and positions. The integral image value at a pixel (x, y) in the image is defined in references [19, 20]:(1)γγx,y=∑x′≤x,y′≤yγx′,y′,where γγx,y is the integral image and γx,y is the original image. We pass the following iteration formula [21, 22]:(2)sx,y=sx,y−1+γx,y,γγx,y=γγx−1,y+sx,y.Once the image is scanned, when the pixel in the lower right corner of the image is reached, the constructed image is shown in Figure2:Figure 2
Integral image calculation demonstration.Wherein, in the integral image, point a represents the sum of the pixel values of area A, pointb represents the sum of the pixel values of the two areas A and B, point c represents the sum of the pixel values of the two areas A and C, and point d represents the sum of the pixel values of the two areas A and C. The sum of the pixel values of the four regions A, B, C, and D. Therefore, any rectangle can be derived from the integral of the 8 rectangular corner points [23, 24].The AdaBoost algorithm trains the identical narrow classifier on separate practice episodes. The partial categories acquired from separate practice episodes are then combined to form a definitive and powerful categorizer. The AdaBoost algorithm is used because the Haar-like features have high dimensionality and it is necessary to choose them with a particular method and incorporate them into a powerful categorizer to examine a human face [25].(3)hx=0,other,1,∑t=1Tathtx≥12∑t=1Tat.


### 2.3. Emoticon Image Preprocessing

Geometric normalization. The MTCNN algorithm is used to detect the position of the face and extract the key points of the face. The center point of each eye is calculated according to the above-mentioned face key points and connection, and the angle between the horizontal line is calculated. According to this angle, we affine the picture. Thereby, the horizontally aligned picture can be obtained. At this time, the face is cut according to a certain scale factor, and the degree change and the face rotation can be processed by the abovementioned method. Grayscale normalization. The picture pixels are normalized to [0, 1], mainly to reduce image quality problems caused by illumination changes.


### 2.4. Training Deep Convolutional Neural Network

The normal decomposition paradigm employed in building deep decomposed recurrent nets commonly consists of three primary levels, including a decomposition level, a pooling level, and a wholly concatenated level. Different layers have different functions. A neural network is capable of image recognition, its input has a lot of pixels. The first layer of computing units can only recognize a few shapes, such as the difference between light and dark, and the second layer of neurons is transmitted to the first layer of neurons As a result, the analysis obtains information such as edges and angles, and continues abstracting layer by layer and eventually obtaining high-level features that are closer to the nature of the research object [26].Given the input, the calculation process for the entire network is as follows:For convolutional layer inputX, the convolution process is as follows [27, 28]:(4)con=f∑i,j∈Mxi,jwm−i,n−j+b,where x represents the element in the convolutional region of input X, w represents the element in the convolution kernel, m,n represents the size of the convolution kernel, b represents the offset, and f represents the ReLu activation function.For the pooling layer inputy, the pooling process is [9](5)pool=downmaxyi,j,i,j∈p,where y represents the pooling layer input element, and down is the downsampling process, which retains the maximum value in the pooled area.For the fully connected layer inputG, the formula is as follows [29]:(6)full=fw∗z+b,where z represents the input element, w represents the weight, b represents the offset, and f represents the ReLu activation function.For the input layer input, the formula is as follows [30]:(7)py=jx;w=ewlx∑l=1Kewlx.


### 2.5. Expression Classification

This chapter uses SoftMax as the expression recognition classifier. We set the training set tox1,y1,⋯,xm,ym with input features xi∈Rn+1, and the class is labeled yi∈1,2,k,⋯7:(8)hθx=pyi=1xi;θpyi=2xi;θ⋮pyi=kxi;θ=1∑j=1keθJTxieθ1Txieθ2Txi⋮eθkTxi,where pyi=jxi represents the probability θ1,θ2,⋯θk∈Rn+1 of the input xi of the ith sample belonging to the category j. The system cost function is as follows [31, 32]:(9)Jθ=−1m∑i=1m∑j=1k1yi=jlogeθjTxi∑i=1keθjTxi.Cost function gradient∇θjJθ:(10)∇θjJθ=−1m∑i=1mxi1yi=j−pyi=jxi;θ.


## 2.1. The Overall Framework of Expression Recognition Based on Deep Convolutional Neural Networks

In this section, a dynamic expression identification system using profound involutional integer systems is designed [16]. The system consists of a computer camera and software function modules. The video image information is collected in real time by the camera in a real environment, and each frame of the video is separately calculated. In the software part, first, the Haar classifier is used for face detection. Second, the obtained face image is preprocessed to obtain the face gray image; the obtained gray image is input to the trained deep convolutional neural network again. Finally, the resulting expression features are entered into the SoftMax classifier for expression classification. Real-time dynamic facial expression recognition [17].The block diagram of the expression recognition system based on deep convolutional neural network is shown in Figure1.Figure 1
Block diagram of the expression recognition system based on the deep convolutional neural network.


## 2.2. Face Detection Using the Haar Classifier

This section uses the face detection method based on the Haar-like feature and the AdaBoost algorithm, namely, the Haar classifier algorithm [18]. This algorithm uses the Viola–Jones target detection framework, which is a real-time object detection method, which can realize the first frame from video. It positions the face to achieve real-time expression recognition. The integral graph algorithm can be used to quickly calculate Haar-like features with different sizes and positions. The integral image value at a pixel (x, y) in the image is defined in references [19, 20]:(1)γγx,y=∑x′≤x,y′≤yγx′,y′,where γγx,y is the integral image and γx,y is the original image. We pass the following iteration formula [21, 22]:(2)sx,y=sx,y−1+γx,y,γγx,y=γγx−1,y+sx,y.Once the image is scanned, when the pixel in the lower right corner of the image is reached, the constructed image is shown in Figure2:Figure 2
Integral image calculation demonstration.Wherein, in the integral image, point a represents the sum of the pixel values of area A, pointb represents the sum of the pixel values of the two areas A and B, point c represents the sum of the pixel values of the two areas A and C, and point d represents the sum of the pixel values of the two areas A and C. The sum of the pixel values of the four regions A, B, C, and D. Therefore, any rectangle can be derived from the integral of the 8 rectangular corner points [23, 24].The AdaBoost algorithm trains the identical narrow classifier on separate practice episodes. The partial categories acquired from separate practice episodes are then combined to form a definitive and powerful categorizer. The AdaBoost algorithm is used because the Haar-like features have high dimensionality and it is necessary to choose them with a particular method and incorporate them into a powerful categorizer to examine a human face [25].(3)hx=0,other,1,∑t=1Tathtx≥12∑t=1Tat.


## 2.3. Emoticon Image Preprocessing

Geometric normalization. The MTCNN algorithm is used to detect the position of the face and extract the key points of the face. The center point of each eye is calculated according to the above-mentioned face key points and connection, and the angle between the horizontal line is calculated. According to this angle, we affine the picture. Thereby, the horizontally aligned picture can be obtained. At this time, the face is cut according to a certain scale factor, and the degree change and the face rotation can be processed by the abovementioned method. Grayscale normalization. The picture pixels are normalized to [0, 1], mainly to reduce image quality problems caused by illumination changes.


## 2.4. Training Deep Convolutional Neural Network

The normal decomposition paradigm employed in building deep decomposed recurrent nets commonly consists of three primary levels, including a decomposition level, a pooling level, and a wholly concatenated level. Different layers have different functions. A neural network is capable of image recognition, its input has a lot of pixels. The first layer of computing units can only recognize a few shapes, such as the difference between light and dark, and the second layer of neurons is transmitted to the first layer of neurons As a result, the analysis obtains information such as edges and angles, and continues abstracting layer by layer and eventually obtaining high-level features that are closer to the nature of the research object [26].Given the input, the calculation process for the entire network is as follows:For convolutional layer inputX, the convolution process is as follows [27, 28]:(4)con=f∑i,j∈Mxi,jwm−i,n−j+b,where x represents the element in the convolutional region of input X, w represents the element in the convolution kernel, m,n represents the size of the convolution kernel, b represents the offset, and f represents the ReLu activation function.For the pooling layer inputy, the pooling process is [9](5)pool=downmaxyi,j,i,j∈p,where y represents the pooling layer input element, and down is the downsampling process, which retains the maximum value in the pooled area.For the fully connected layer inputG, the formula is as follows [29]:(6)full=fw∗z+b,where z represents the input element, w represents the weight, b represents the offset, and f represents the ReLu activation function.For the input layer input, the formula is as follows [30]:(7)py=jx;w=ewlx∑l=1Kewlx.


## 2.5. Expression Classification

This chapter uses SoftMax as the expression recognition classifier. We set the training set tox1,y1,⋯,xm,ym with input features xi∈Rn+1, and the class is labeled yi∈1,2,k,⋯7:(8)hθx=pyi=1xi;θpyi=2xi;θ⋮pyi=kxi;θ=1∑j=1keθJTxieθ1Txieθ2Txi⋮eθkTxi,where pyi=jxi represents the probability θ1,θ2,⋯θk∈Rn+1 of the input xi of the ith sample belonging to the category j. The system cost function is as follows [31, 32]:(9)Jθ=−1m∑i=1m∑j=1k1yi=jlogeθjTxi∑i=1keθjTxi.Cost function gradient∇θjJθ:(10)∇θjJθ=−1m∑i=1mxi1yi=j−pyi=jxi;θ.


## 3. Experiments

In this paper, two expression image libraries are used in the experiment. One is the standard image library JAFFE, and the other is to collect the image library dataset, which is processed on the Internet. JAFFE is a database of expressions established in Japan. It is used by a large number of facial expression recognition researchers and has a certain persuasive power. The JAFFE facial expression image library is compiled by ten Japanese women as models. Each expression selects three to four images according to the size of the amplitude. Each person makes seven expressions to collect, and together, there are 213 images, depending on the resolution of the image. It is 256 × 256. Due to the needs of the experiment, the original pixels in the expression library will be sampled, and image libraries of four different pixels of 16 × 16, 32 × 32, 64 × 64, and 128 × 128 will be created. The comparison of the samples in the JAFFE image library at different resolutions shows that lower the resolution, the more blurred the information carried by the image.The second database, which collected about 13 people, each had 75 emoticons, for a total of 965 images. The image resolution is 64 × 64. Similarly, this paper has established 16 × 16 and 32 × 32 expression libraries, respectively. Due to the mixed expressions in this expression database, a relatively obvious and representative expression image was selected, and a total of six expressions were selected, which were happy, surprised, sad, neutral, angry, and disgusted.The experimental code implementation is in the Python language, the IDE is PyCharm graphical interface tool, the running environment is window7, memory 4G, i5 processor. Since the experiment involved image processing, a large number of matrix operations were required, so the program was selected to run on the GPU. This not only improves the efficiency of the experiment but also shortens the experiment time. Since there are many unpredictable situations during the experiment, in order to ensure the validity of the experimental results, five cross-validation methods were carried out in the experiment of the paper. Here, the image samples of the established facial expression image library are equally divided into 5 for each experiment, 4 sets of images were used for network training, and the rest were used for experimental tests. The process of training to identify such an experiment is repeated five times; then, the average of the five experimental results is the experimental result of the paper.


## 4. Discussion

In this paper, the face image is grayscaled and grayscale normalized before image processing. Figure3 shows the image processing results in this paper, where Figure 3(a) is the color image. Figure 3(b) is the image after the image is grayscaled, and Figure 3(c) is the image after the image grayscale is normalized. It can be seen from the figure that after the image gray processing, there is no change in the overall structure of the image, and it is advantageous to extract the object features for analysis after preprocessing the image.Figure 3
Image preprocessing. (a) Color image. (b) Grayscale image. (c) Normalized image.After preprocessing the image, this section of the experiment constructs the CNN structure shown in Table1 for the original expression set of 32 × 32 and 64 × 64 resolutions, respectively. The learning rate setting includes setting the fixed learning rate, the unified local optimal learning rate, and layering. The dynamic local optimal learning rate is trained in three scenarios, and the network convergence comparison is shown in Figures 4 and 5.Table 1
CNN structure description.
ResolutionNetwork structureConvolution kernelActivation function32 × 32C6S6C13S135, 5RReLU + sigmoid64 × 64C6S6C9S9C12S129, 5, 5RReLU + sigmoidFigure 4
Convergence at 32 × 32 using different learning rate schemes.Figure 5
Convergence on different 64 × 64 using different learning rate schemes.From the experimental results of Figures4 and 5, it can be known that this method achieves similar improvement effects in the training of image samples of 32 × 32 and 64 × 64 images with different resolutions. In general, the convergence speed of uniformly setting the local optimal learning rate is higher than the convergence speed of setting the fixed learning rate, and the convergence speed of the dynamic smooth local optimal learning rate adjustment is significantly faster than the convergence speed of uniformly setting the local optimal learning rate. The influence of various setting methods on the convergence speed is mainly reflected in the early stage. This is because the pretraining period is far from the extreme point, which makes the adjustment of the optimal learning rate show obvious advantages. The learning rate optimization accelerates the speed of network convergence and verifies that the proposed method is feasible for the shallow layer enhancement scheme, and the expression classification accuracy based on the method is higher. This dynamic learning rate adjustment method is used to jump out of the local extremum. Points have a beneficial effect.The learning rate optimization method in this paper brings a certain performance improvement, but it also has defects: the convolutional neural network contains a large number of parameters and calculating the optimal learning rate will incur a certain time cost. If it is optimized for each iteration, it will be optimized once. However, in the case of large data volumes or complex networks, the time loss caused by this will offset the advantage of its convergence speed. In the actual training, the learning rate is updated with the inertia of the learning rate. The shorter the interval between the learning rate adjustment and the longer the convergence step of the network is, the longer the learning rate adjustment interval is, the lower the iterative average time cost is. Choosing the appropriate learning rate adjustment interval requires finding a balance of time performance.For the purpose of ensuring the credibility of the experimental outcomes and preventing the influence of chance agents, the data in the dataset were separated into components and the cross-validation method was adopted. In this experiment, the images in the JAFFE photoset are separated into 3 sections, 2 of which are used as exercise figures as well as the other section as test figures; the images in the dataset are separated into 5 sections, 4 of which will be used as exercise figures each time. One served as the experimental figure. Table2 demonstrates the classification performance of the algorithm for different expressions in the JAFFE representation database. Table 3 demonstrates the categorization outcomes for various representations in the dataset drainage system database. It is obvious from Tables 2 and 3. In some test sets, the whole network performs better and the correct rate is high, but it is relatively poor in others. The explanation for this is probably the expressive traits that can be derived from the exercise model material. Inadequate access to sufficient training features results in a poor classification of the classifier, a situation that mirrors the principle of the significance of example material for involutional systems.Table 2
Classification accuracy of different expressions of JAFFE expression library (%).
AngryDisgustScaredHappyNeutralSadSurprisedOverall (%)Test set 11008010010010090.9188.8994.37Test set 2100909081.8210010010092.96Test set 310010081.8290.9110010010095.77Overall (%)10089.6690.6390.6310096.7796.5594.37Table 3
Classification accuracy of different expressions of dataset database (%).
AngryDisgustScaredHappyNeutralSadSurprisedOverall (%)Test set 188.8994.448092.8670.839693.9488.89Test set 270.3777.788096.30688496.9782.32Test set 377.7885.7184.62100647293.9483.33Test set 462.9694.298889.29608287.8880.81Test set 584.4885.717292.866479.1710083.33Overall (%)76.3087.5980.9594.2665.3782.2394.5583.75First, the original expression set is balanced then, different degrees of amplification are obtained to obtain a plurality of amplified expression sets, and then the amplified expression set is trained using the optimal model determined in the previous section experiment, and some samples are subjected to (−10, the rotation of the −5, 5, and 10) degree, and the mirror flip process amplifies the sample to 151,974 sheets. The network trained a total of 1,000 rounds, with the training batch size set to 512. In training, 75% of the total sample was randomly selected as the training sample, and the remaining 25% was used as the test sample. The results of statistical experiments were compared to compare the classification accuracy and generalization ability of models based on different amplification sets. In this section, the data balance uses the SMOTE algorithm to linearly interpolate a small number of dense regions of the original expression set. The processed expression set contains a total of 3,500 expression images, and the seven types of expression samples are balanced and each has 500. It is further hoped to improve the generalization ability of the model. In this paper, the existing samples of the balanced expression set are randomly rotated and distorted to generate new samples, and the new samples are added to the existing training set to obtain the amplification set. According to the need for experimental comparison, the CNN models established in the previous section were used to train a plurality of different degrees of amplification sets, and the ability to obtain the expression classification is shown in Table4. Generalization ability of the amplification set. Figure 6 shows the simulation results of 35,000 iterations (about 470 rounds). On the whole, the loss on the test set gradually decreases with the number of iterations. After 35,000 times, the loss tends to be stable at around 0.2 and no longer decreases. The accuracy rate on the test set gradually increases with the number of iterations. Through gradient learning and continuous learning and updating of the model weight parameters, the classification ability of the network is getting stronger and stronger, and the accuracy of the final test set is basically stable.Table 4
Expression recognition using different amplification ratio data.
Amplification factorInput map resolutionClassification accuracy (amplification) (%)132 × 3298.7464 × 6498.71232 × 3298.5264 × 6498.76332 × 3298.7664 × 6498.87432 × 3298.8464 × 6498.90532 × 3298.8564 × 6498.91632 × 3298.8564 × 6498.90Figure 6
Accuracy and loss trend curve of training on the database.Data amplification is beneficial to improve the generalization ability of the model, and the richer the amplified data, the stronger the generalization ability, because: The CNN structure itself is a complex network with a large number of parameters, and the richness of the dataset can alleviate its overlearning of existing samples. The random data amplification method in this paper differentiates the information contained in the same kind of expression, which is beneficial to the model because enables it to pay more attention to the common features in the process of fitting the data. In addition, as can be seen from Table2, the data preprocessing in this paper has a positive impact on the expression classification ability, but the improvement of the classification ability does not increase linearly with the amplification multiple, because the data amplification is only limited to the existing data. A manual processing method is not a panacea, and the quality of the original data is often the bottleneck of the final recognition performance.


## 5. Conclusions

In today’s society, facial expressions are a popular form of expressing affective messages in relation to public events. Facial expressions contain a diverse collection of signals, including an individual’s affective mood state, individual traits, and perceptions of events. In addition, numerous elements relating to an individual’s state of mood, mental conditions, and physiological status are also relevant to the messages contained in embedded visual representations. Psychologist Mehrabian proposed that in the process of human communication, only 7% of the information is expressed by language. Compared with this, facial expression is 55% of the information, which is the most of the heavy part of the information. It can be seen that the research on the facial expression recognition technology of the intelligent service robot of the nanofiber flexible wearable electronic skin can help us obtain a lot of valuable information and then analyze and study the human psychological and physiological state, thereby improving people’s living standards.In order to achieve the purpose of quantitative analysis of human emotions, research studies on human expression recognition technology is the key to success. At the same time, expression recognition plays an indispensable role in intelligent robots and smart cities. With the continuous development of artificial intelligence and machine learning technology, the demand for intelligent human-computer interaction in society is also expanding. With the deepening of research, many outstanding image open-source frameworks and high-performance hardware devices have provided good technical support for the development of facial expression recognition. At present, the expression recognition technology of intelligent service robots with nanofiber flexible wearable electronic skin is not a unique research topic in the field of computer vision but a cross-domain topic in many fields such as biology and psychology. The research results have important implications for many fields, such as human-computer interaction, healthcare, and safe driving.Although this paper uses the intelligent service robot facial expression recognition technology to conduct a profound study on the wearable electronic skin of nano-flexible fibers, there are still many deficiencies. The depth and breadth of the research in this paper is not enough. Moreover, the acquisition is carried out under absolutely ideal conditions, and the completeness and effectiveness are not enough. In future works, we will study suitable methods and means from more angles based on the existing technology and level and continuously improve the recognition quality.



---

*Source: 8669298-2022-09-30.xml*