# A 3D Face Modeling and Recognition Method Based on Binocular Stereo Vision and Depth-Sensing Detection

Authors: Ziyou Zhang; Ziliang Feng; Wei Wang; Yanqiong Guo
Journal: Journal of Sensors
 (2022)

---

## Abstract

The human face is an important channel for human interaction and is the most expressive part of the human body with personalized and diverse characteristics. To improve the modeling speed as well as recognition speed and accuracy of 3D faces, this paper proposes a laser scanning binocular stereo vision imaging method based on binocular stereo vision and depth-sensing detection method. Existing matching methods have a weak anti-interference capability, cannot identify the spacing of objects quickly and efficiently, and have too large errors. Using the technique mentioned in this paper can remedy these shortcomings by mainly using the laser lines scanned onto the object as strong feature cues for left and right views for binocular vision matching and thus for depth measurement perception of the object. The experiments in this paper first explore the effect of parameter variation of the laser scanning binocular vision imaging system on the accuracy of object measurement depth values in an indoor environment; the experimental data are selected from 68 face feature points for modeling, cameras that photograph faces, stereo calibration of the cameras using calibration methods to obtain the parameters of the binocular vision imaging system, and after stereo correction of the left and right camera images, the laser line scan light is used as a pixel matching strong features for binocular camera matching, which can achieve fast recognition and high accuracy and then select the system parameters with the highest accuracy to perform 3D reconstruction experiments on actual target objects in the indoor environment to achieve faster recognition.


---

## Body

## 1. Introduction

Reproduction of realistic human faces has been a long-standing pursuit. Early methods often used include hand painting, clay sculpture, and carving. With the development of technology, photographic methods emerged to reproduce real faces, but they were limited to reflecting the two-dimensional information of a face at a certain angle. Since the middle of the twentieth century, computer technology has developed rapidly, and more and more research has been conducted on the use of computers to synthesize realistic facial models, and fruitful results have been achieved [1]. Nowadays, face modeling and animation have become a hot research topic in the field of computer graphics and have received widespread attention [2], and market demand is the driving force for the continuous development of face modeling and animation research. At present, the 3D face model has been initially applied to video telephony, video conference, film and television production, computer games, face recognition, and other fields, and with the development of various technologies, 3D face models will be more deeply and widely used in all aspects.Binocular stereo vision to restore the three-dimensional method is analogous to the human eyes to obtain two-dimensional information about objects in the line of sight, information after the retina and other organs to the central system for analysis and processing, and finally reveal the three-dimensional information in the field of vision method, which is often referred to as “bionics” in the scientific community. Through the above bionic principle, the binocular camera system simulates the human binocular system to obtain the depth of the scene. From the camera side, two two-dimensional images of the same scene are acquired from different viewpoints, and the difference of the same point in thex-direction of the horizontal coordinate between the left and right cameras is calculated using the relevant stereo matching algorithm, i.e., the parallax value [3]. Finally, the depth information of the object is calculated by the perspective change, and the world coordinates of the object in the field of view concerning the left camera are obtained by combining the parameters obtained from the camera calibration. The process of transforming any point in space into the two-dimensional plane of the image can be done by the camera, and this process of change is essentially the evolution of a geometric model. There are a wide variety of models, the simplest and most widely used being the pinhole model. The model describes the relationship between a beam of light passing through a pinhole and the image projected on the back of the pinhole.Virtual reality technology has four main characteristics: interactivity, multiperception, presence, and autonomy. The interactivity includes two main aspects: on the one hand, the process of user manipulation of objects in the virtual environment, and, on the other hand, the feedback effect of the manipulated objects on the user [4]. It is well known that in the real world, the perceived righteousness of nature and the sense of the existence of man himself are dependent on the interaction process between man and nature, where man exerts his subjective initiative to transform nature, along with the transformation, process nature gives diverse feedback to man [5]. Virtual reality is a simulation of the natural world, so interactivity plays a pivotal role in virtual reality technology. Therefore, the 3D face modeling method based on binocular stereo vision and depth-sensing detection is not only faster in modeling but also very fast for face recognition.


## 2. Related Works

Liu [6] was the first researcher to construct face models by computer, specifically by using primary polygons to identify the human head and successfully implementing the first parametric model of the human face. Since Xue [7] proposed parametric models of faces in the 1970s, 3D face technology has developed rapidly in the following decades. The traditional 3D face modeling method is mainly manual modeling, such as using Maya, 3DMax, and other 3D modeling software to realize the model construction, which was the mainstream modeling method in the early days and required modelers to be proficient in modeling software and have professional modeling experience. At the same time, modeling work is a job that requires time and effort, which puts high demands on the professionalism of modelers. The inconvenience of manual modeling and the high demand for 3D models have given rise to the research and development of 3D imager and image-based 3D modeling technology. With the depth of related research, there are three more mature ways to acquire 3D models of faces, based on structured light, based on 3D laser scanning technology, and based on image modeling acquisition. In his study, the literature used color invariants and sequential visibility rules to reconstruct the scene model, which was automated and did not require human interaction, and noted that the results of this algorithm have been able to accurately and reliably approach the results of 3D scanners. Zhang used four high-resolution images for the model construction of the face, while the algorithm used was a multiangle reconstruction algorithm based on mesh, and from the results, the comparison between the model built and the real face has reached a realistic level. A high-resolution face reconstruction method with detailed geometry and high-fidelity texture is proposed in Li et al.’s study [8], which is rarely applicable to the reconstruction of more complex textured faces (e.g., more wrinkles), although there is some improvement in accuracy. Shading-based reconstruction method SFS (Shape From Shading) reconstructs from surface texture information by analyzing the brightness and shading of images under different lighting conditions with an albedo lighting model. Zhang [9] proposed a combination of feature extraction and shading structure, which achieved good results. Although a more accurate 3D face model can be obtained from a single image, there are some drawbacks, and the brightness of image pixels is constrained by many issues such as light source index, camera parameters, and the distribution of the target face surface.Zhou [10] starts from a single side-view face image, derives the feature points of the front-view face, calculates the corresponding face feature points of the symmetric view according to the symmetry, and then uses the SFM algorithm to reconstruct the sparse 3D model. Although the effect caused by feature point occlusion is eliminated, the alignment error will be caused by inaccurate positioning. He [11] proposes a modeling method based on symmetric side views and then uses motion recovery structures for sparse and dense model reconstruction. However, this process is too complex and the point cloud data operation and modeling steps have high complexity, which is not conducive to the subsequent product production and cannot be used as the preferred method for 3D face model generation. Shan et al. [12] introduced a VMS method with an orthogonal chunking model, which achieved very good results in high-resolution image benchmarking. Yu [13] utilized a pixel-based VMS method to derive a dense 3D face model, which consists of three main steps: first, identifying the features corresponding to multiple stereo faces [14].However, in contrast, the reconstruction process suffers from underfitting due to the reduced number of feature points. Dawar and Kehtarnavaz [15] use a combination of sparse point SFM and standard face model for modeling, which results in an underfitting of the face model due to the reduced amount of information. Moeinkhah [16] does not consider adding features before and texture before modeling and only redescribes the energy function starting from the facial shadow constraint, which equates to the insufficient refinement of the model and low model accuracy. Ximenes [17] used a cascade regression framework to estimate 3DMM parameters, but the final model accuracy was not high because the feature points were random. De Chaumont [18] combines the shape prior, landmark, contour, and color-texture difference domains of the face using a competitive training approach that combines a large number of feature points, leading to a very high computational complexity of the fitting process, which is not conducive to face model reconstruction. The most common statistical model currently used for 3D face reconstruction is called 3D Morphable Model (3DMM) [19], which uses standard principal component analysis to analyze the shape and texture of faces in a 3D face database to fit parameters based on the features extracted from the input image, and the fitting process is highly dependent on the number of features and nonlinear optimization algorithms.


## 3. The Implementation of 3D Face Modeling

### 3.1. Face Feature Point Extraction Algorithm

Common three-dimensional face acquisition is the use of a three-dimensional laser scanner to collect, which requires a large economic cost, a machine in the tens of millions. Therefore, it is a good choice to reconstruct the three-dimensional information of the face by extracting the feature points in the two-dimensional image. In this paper [20], multiple cameras are used for face information acquisition, and the 3D model is constructed from the acquired 2D images. In this paper, multiple cameras are evenly distributed around the scene during acquisition, and the cameras have a certain angle with the horizontal direction of the scene [21]. When multiple cameras are used for simultaneous acquisition, the frame rate and resolution of each camera should be the same for the cameras to process the information synchronously. The data collected by each camera is sent to the computer through the network. As the basis of the face rough model, the transformation of face feature points is different from rigid objects such as buildings, which contain not only rigid deformation regions, i.e., [22] affine transformation regions, but also nonrigid deformation regions. This chapter uses a new face-oriented feature extraction and alignment approach to optimize the adjustment results of multiple variables, treating the affine transformation and the nonrigid transformation as two independent phases. In the affine transformation stage, the key points of each face are divided into five groups and each group is affine transformed separately. In the nonrigid transformation stage, all key points of the face will be considered and the points of the overall face will be optimized to finally obtain the exact face feature points, and then, the rough face model will be obtained [23]. The principle is shown in Figure 1.Figure 1
Face rough model generation schematic.3D face modeling is mainly based on 2D images or video deformation of standard models to build 3D face models, while 3D models can also be built directly using 3D scanner scanning methods. Most of the current research on 3D modeling is aimed at the former firstly introducing the algorithm of face feature point extraction. Since the acquisition method is not uniform, face facial feature extraction methods have received extensive attention from the academic community [24]; the classical methods include the local constraint model (CLM), convolutional neural network (CNN), and deep learning methods; supervised descent method (SDM) belongs to a class of regression methods; it is achieved by the iterative way, the principle of a given face. The initial state on the image is regressed to the real face shape by iteratively aligning the image to a specific template without learning any model of the shape or appearance from the training data in advance. For face shapes, SDM will directly extract the locations of feature points since it is a nonparametric model. The first regression input at this moment is the SIFT features around the initial feature points, and since the SIFT operator is not derivable, the only way to minimize the above objective function using a first- or second-order method is to estimate the Hessian matrix using a numerical approximation, which is expressed as
(1)A=μX+∑i=1nXiYi+x−μσ.The error decreases monotonically with the number of regressions added. In the feature extraction experiments, the SDM eventually converges in 4 or 5 steps. Thus, the SDM-based feature extraction algorithm is derived, and the learning and testing process is shown in Figure2.Figure 2
SDM regression learning process and testing process.


### 3.2. Binocular Stereo Vision Algorithm for 3D Modeling of the Human Face

In 3D reconstruction and face modeling, binocular stereo vision has been used more often, which can quickly acquire 3D coordinates and does not require human intervention, so it has better results in 3D face recognition and reconstruction. However, in the face modeling process, some face regions do not have obvious features and are difficult to recognize, so it is not only difficult to match effectively by directly applying the grayscale similarity principle [25] but also difficult to create models, so this paper uses two different methods, specifically: the extraction of the main feature points of the left and right faces is completed by the ASM algorithm, the initial parallax map is clarified, and then, the adaptive parallax map is improved and adjusted by using the filtering algorithm. The feature points obtained by the extraction algorithm may have offset coordinates and inaccurate feature point positions, and face alignment can be used as a method to optimize the facial feature extraction results. Its mathematical formula is
(2)A=∑i=1nXi2+σX2+∑i=1nXi−X¯2.Face alignment is first performed across multiple face faces, and then, constraints between multiple face faces are used to further optimize the alignment results. Both feature alignment and local face alignment for unconstrained faces have received a lot of attention, and each method can achieve both constrained and unconstrained face alignment. Its mathematical formula is(3)M=σX+∑i=1nXiYi.For the self-collected face database in this paper, since it is a face image collected under the same camera parameters and the same three-view angle, it belongs to the constrained face, and the face feature alignment algorithm can be achieved as well. The problem about face feature point extraction and alignment is transformed into a problem of solving a set of parameters using regression, which in turn yields 68 feature points of the face. Its mathematical formula is(4)N=1n∑i=1nXiYi+dydx+∂2Ω∂u2.By combining face feature point extraction with grouped face feature alignment and dividing the face feature points into small-scale point sets, a more accurate face with 68 feature points can be obtained, thus laying a good foundation for the deformation of the rough model. The schematic diagram of its face 3D modeling process is shown in Figure3.Figure 3
Schematic diagram of face 3D modeling process.


### 3.3. Face Matching Process for the Depth-Sensing Detection Algorithm

After the face image correction process is done, the feature points of the left and right faces are first extracted by the depth-sensing detection algorithm, and the parallax search area is initially determined by its feature points. In this paper, the depth-sensing detection algorithm is used for the matching search because this algorithm does not need to decide the window size value based on personal experience, and its matching result is ideal in the edge region; in addition, the subpixel interpolation operation is implemented here to obtain more accurate parallax values. After obtaining the initial parallax map of the left and right images, a consistency check is unfolded to remove all the points that are inconsistent between left and right.The bilateral filtering method (bilateral filtering) is proposed based on the Gauss filtering method, mainly for Gauss. Its mathematical formula is(5)Fx=μX+∑i=1nXiYi+x−μσ.The principle of image filtering by directly convolving Gauss weight coefficients with image information is to optimize the filter weight coefficients as the product of Gauss function and image brightness information and then convolve the optimized weight coefficients with image information so that the image edge information in the image information can be taken into account while filtering, and the blurred edge information of the image after normal Gauss filtering can be the edges of the image that are blurred after normal Gauss filtering are kept clear and the edges of the image are smoother. Finally, the median filtering method is used to obtain not only the parallax map but also the face depth map successfully recovered. The initial matched results need to be further processed; i.e., all incorrect matching results need to be removed to improve the accuracy of the results. Subpixel interpolation is applied to the initial matching results. It is a compromise processing that combines the spatial proximity of the image and the similarity of the pixel value and considers the spatial information and grayscale similarity at the same time, to achieve the purpose of edge-preserving and denoising. It is simple, noniterative, and local. The advantage of the bilateral filter is that it can be used for edge preservation. Generally, the Wiener filter or Gaussian filter used in the past to denoise will blur the edges more obviously, and the protective effect of high-frequency details is not obvious. The reason for this is that when stereo matching is performed, the matching results are all integer values, and this places a constraint on the continuity of the reconstructed depth map. As shown in Figure4, the minimum matching surrogate value is selected as an example, and the lowest value is considered as the final parallax value by parabolic interpolation.Figure 4
Principle of the depth-sensing detection algorithm.


## 3.1. Face Feature Point Extraction Algorithm

Common three-dimensional face acquisition is the use of a three-dimensional laser scanner to collect, which requires a large economic cost, a machine in the tens of millions. Therefore, it is a good choice to reconstruct the three-dimensional information of the face by extracting the feature points in the two-dimensional image. In this paper [20], multiple cameras are used for face information acquisition, and the 3D model is constructed from the acquired 2D images. In this paper, multiple cameras are evenly distributed around the scene during acquisition, and the cameras have a certain angle with the horizontal direction of the scene [21]. When multiple cameras are used for simultaneous acquisition, the frame rate and resolution of each camera should be the same for the cameras to process the information synchronously. The data collected by each camera is sent to the computer through the network. As the basis of the face rough model, the transformation of face feature points is different from rigid objects such as buildings, which contain not only rigid deformation regions, i.e., [22] affine transformation regions, but also nonrigid deformation regions. This chapter uses a new face-oriented feature extraction and alignment approach to optimize the adjustment results of multiple variables, treating the affine transformation and the nonrigid transformation as two independent phases. In the affine transformation stage, the key points of each face are divided into five groups and each group is affine transformed separately. In the nonrigid transformation stage, all key points of the face will be considered and the points of the overall face will be optimized to finally obtain the exact face feature points, and then, the rough face model will be obtained [23]. The principle is shown in Figure 1.Figure 1
Face rough model generation schematic.3D face modeling is mainly based on 2D images or video deformation of standard models to build 3D face models, while 3D models can also be built directly using 3D scanner scanning methods. Most of the current research on 3D modeling is aimed at the former firstly introducing the algorithm of face feature point extraction. Since the acquisition method is not uniform, face facial feature extraction methods have received extensive attention from the academic community [24]; the classical methods include the local constraint model (CLM), convolutional neural network (CNN), and deep learning methods; supervised descent method (SDM) belongs to a class of regression methods; it is achieved by the iterative way, the principle of a given face. The initial state on the image is regressed to the real face shape by iteratively aligning the image to a specific template without learning any model of the shape or appearance from the training data in advance. For face shapes, SDM will directly extract the locations of feature points since it is a nonparametric model. The first regression input at this moment is the SIFT features around the initial feature points, and since the SIFT operator is not derivable, the only way to minimize the above objective function using a first- or second-order method is to estimate the Hessian matrix using a numerical approximation, which is expressed as
(1)A=μX+∑i=1nXiYi+x−μσ.The error decreases monotonically with the number of regressions added. In the feature extraction experiments, the SDM eventually converges in 4 or 5 steps. Thus, the SDM-based feature extraction algorithm is derived, and the learning and testing process is shown in Figure2.Figure 2
SDM regression learning process and testing process.


## 3.2. Binocular Stereo Vision Algorithm for 3D Modeling of the Human Face

In 3D reconstruction and face modeling, binocular stereo vision has been used more often, which can quickly acquire 3D coordinates and does not require human intervention, so it has better results in 3D face recognition and reconstruction. However, in the face modeling process, some face regions do not have obvious features and are difficult to recognize, so it is not only difficult to match effectively by directly applying the grayscale similarity principle [25] but also difficult to create models, so this paper uses two different methods, specifically: the extraction of the main feature points of the left and right faces is completed by the ASM algorithm, the initial parallax map is clarified, and then, the adaptive parallax map is improved and adjusted by using the filtering algorithm. The feature points obtained by the extraction algorithm may have offset coordinates and inaccurate feature point positions, and face alignment can be used as a method to optimize the facial feature extraction results. Its mathematical formula is
(2)A=∑i=1nXi2+σX2+∑i=1nXi−X¯2.Face alignment is first performed across multiple face faces, and then, constraints between multiple face faces are used to further optimize the alignment results. Both feature alignment and local face alignment for unconstrained faces have received a lot of attention, and each method can achieve both constrained and unconstrained face alignment. Its mathematical formula is(3)M=σX+∑i=1nXiYi.For the self-collected face database in this paper, since it is a face image collected under the same camera parameters and the same three-view angle, it belongs to the constrained face, and the face feature alignment algorithm can be achieved as well. The problem about face feature point extraction and alignment is transformed into a problem of solving a set of parameters using regression, which in turn yields 68 feature points of the face. Its mathematical formula is(4)N=1n∑i=1nXiYi+dydx+∂2Ω∂u2.By combining face feature point extraction with grouped face feature alignment and dividing the face feature points into small-scale point sets, a more accurate face with 68 feature points can be obtained, thus laying a good foundation for the deformation of the rough model. The schematic diagram of its face 3D modeling process is shown in Figure3.Figure 3
Schematic diagram of face 3D modeling process.


## 3.3. Face Matching Process for the Depth-Sensing Detection Algorithm

After the face image correction process is done, the feature points of the left and right faces are first extracted by the depth-sensing detection algorithm, and the parallax search area is initially determined by its feature points. In this paper, the depth-sensing detection algorithm is used for the matching search because this algorithm does not need to decide the window size value based on personal experience, and its matching result is ideal in the edge region; in addition, the subpixel interpolation operation is implemented here to obtain more accurate parallax values. After obtaining the initial parallax map of the left and right images, a consistency check is unfolded to remove all the points that are inconsistent between left and right.The bilateral filtering method (bilateral filtering) is proposed based on the Gauss filtering method, mainly for Gauss. Its mathematical formula is(5)Fx=μX+∑i=1nXiYi+x−μσ.The principle of image filtering by directly convolving Gauss weight coefficients with image information is to optimize the filter weight coefficients as the product of Gauss function and image brightness information and then convolve the optimized weight coefficients with image information so that the image edge information in the image information can be taken into account while filtering, and the blurred edge information of the image after normal Gauss filtering can be the edges of the image that are blurred after normal Gauss filtering are kept clear and the edges of the image are smoother. Finally, the median filtering method is used to obtain not only the parallax map but also the face depth map successfully recovered. The initial matched results need to be further processed; i.e., all incorrect matching results need to be removed to improve the accuracy of the results. Subpixel interpolation is applied to the initial matching results. It is a compromise processing that combines the spatial proximity of the image and the similarity of the pixel value and considers the spatial information and grayscale similarity at the same time, to achieve the purpose of edge-preserving and denoising. It is simple, noniterative, and local. The advantage of the bilateral filter is that it can be used for edge preservation. Generally, the Wiener filter or Gaussian filter used in the past to denoise will blur the edges more obviously, and the protective effect of high-frequency details is not obvious. The reason for this is that when stereo matching is performed, the matching results are all integer values, and this places a constraint on the continuity of the reconstructed depth map. As shown in Figure4, the minimum matching surrogate value is selected as an example, and the lowest value is considered as the final parallax value by parabolic interpolation.Figure 4
Principle of the depth-sensing detection algorithm.


## 4. Experimental Design

This chapter is an experiment and analysis of hardware system design for indoor and outdoor environments, respectively. The red visible laser scanner is used first. The red visible laser is affected by the visible spectrum band. It is not able to provide effective stereo matching of strong features in different natural light environments outdoors. At the same time, the red visible laser can cause damage to human eyes. So this chapter uses an infrared laser instead of a red visible laser and recaptures images indoors and outdoors. The corresponding binocular camera was also replaced with a camera matching the infrared laser and replaced with a monochromatic camera, which is in line with the original purpose of the research of the subject, to be able to feel the laser bright lines to achieve the fast stereo matching and also to feel the visual image content and provide more semantic information.This chapter analyzes the experimental results of visible light by changing the experimental equipment and the experimental environment for more accurate measurements, changing the laser and the camera, and purchasing an infrared filter that is fixed in front of the lens. An infrared filter is an optical material that shields visible light and transmits only infrared light, which can filter out the excess light and accurately capture the infrared laser lines that hit the object in front of the lens, facilitating subsequent image processing, so that the 3D coordinates can be obtained faster. The specific parameters of the experiment are shown in Table1.Table 1
Experimental hardware parameters.
LaserWavelengthPowerLaser rulerVoltagePw650650 nm60 mW16∗70mmDC3-5 VFu850630 nm40 mW10∗25mmDC2.5-5.9 V


## 5. Analysis of Results

### 5.1. Analysis of Face Feature Point Data Collection

To test the effectiveness of the algorithm, the following two steps are used to test and compare the effect of 3D face reconstruction. In the first step, the 3D face database is directly used, projected as left and right images, matched by the algorithm proposed in this paper, and compared with the actual parallax map; in the second step, the 3D face reconstruction experiment is carried out, and the 3D face reconstruction is completed by the self-designed stereo vision system. In the process of algorithm design, this paper combined the matching cost with the adaptive weighting, and in the database test, the left and right images of the 3D FRD projection of Texas were processed. The efficiency of face feature point acquisition using binocular stereo vision-based depth-sensing detection compared with the traditional method is shown in Figure5.Figure 5
Comparison of the efficiency of binocular stereo vision and depth-sensing detection with the traditional way to identify feature points.The above experiments confirm that the dense matching algorithm can indeed be applied in face matching, and the following paper needs to validate the method proposed autonomously in this paper to see if it can improve in terms of time and error. In practical application scenarios, it is not possible to know the parallax range of a specific object in advance, so a larger range is usually chosen to ensure that the corresponding match point can be found. However, this not only prolongs the search time but also makes it very easy to have a false match. However, the face features are common, and the parallax of its feature points can be explored first so that the maximum and minimum parallax can be clarified, which is helpful to get the parallax accurately in a shorter period.


### 5.2. Analysis of the Efficiency of 3D Face Modeling

This experiment is divided into 5 groups: first, the first group according to the actual maximum parallax and minimum parallax to determine the search interval, the other four groups in the matching, the maximum parallax, and minimum parallax following the gradient of 5 pixels to expand the standard, the matching time and error of these five groups for comparative analysis, each group need to identify 50 images, where the longer the matching time of binocular stereo vision and depth-sensing detection algorithm, the longer the matching time of binocular stereo vision and depth-sensing detection algorithm, the larger the search interval. The specific matching effects of the four matching algorithms were tested, and some parameters of the algorithms were adjusted. The validation results show that the proposed method has made some progress in both time and error. This section concludes by applying the matching algorithms to the face images obtained through the binocular vision system.The results confirm that the reconstruction effect is eligible, a certain number of models are selected, the worst 5 cases of each method are eliminated, 2 models with larger differences are used for each group to evaluate, and the distribution of the average normalized NME about feature points is calculated for 10 groups.The same number of 68 points is extracted in the same order. And what can be seen is that the extracted feature pixels located at the facial contour, eye boundary, eyebrow boundary, mouth, and nose boundary cover all the important parts of the human face, which are completely accurate to represent the personalized human face. Therefore, the 68 feature point coordinate positions are used as a feature prior constraints, and then, the accurate 3D face model contour estimation is performed, i.e., model refinement, global optimization, and texture colorization. The feature prior constraint is considered an accurate estimation of the face 3D facial model, but its number of features is after all limited and there is no color pixel information. Therefore, there is not enough information to refine the nonfeature regions except for the overall face model which is more obvious. In contrast, by comparing the texture differences of matching pixels between two images, the detailed information of the face can be mined in-depth, but in texture-free regions and repeated texture regions, these algorithms are often unable to reconstruct 3D face models with high precision because there is not enough visual information to be acquired. Fully mining the 3D information implied between multiview images and introducing adaptive texture constraints can improve the accuracy of the estimated 3D face models. Experiments demonstrate that the use of binocular stereo vision with depth-sensing detection algorithms has a significant effect improvement compared with traditional modeling methods, and the efficiency comparison is shown in Figure6.Figure 6
Comparison of binocular stereo vision and depth-sensing detection algorithms with conventional modeling efficiency.


### 5.3. Face Recognition Analysis

In the so-called stereo matching algorithm, that is, a point in the physical world is mapped to the projection point matching process in the left and right imaging pictures of two binocular cameras, the usual process is first a point selection in the image of one camera and then according to the coordinates of this point in the picture of the other camera to match the corresponding point search. If it is based on the polar line constraint, the search can be matched on the same line in the same vertical coordinate, which can reduce the search range greatly. However, there are some other problems and challenges beyond solving the search range only, for example, the same physical area in the photos of 2 cameras due to the different direction and intensity of lighting, and also intermingled with noise effects so that the grayscale color of the matching corresponding points of 2 different cameras at the same point is very different; it is also possible that directly due to the different angles that a point is present in one camera, there is no corresponding point in the photo of the other camera; there are also images due to the repetition of feature points so that the photos of both cameras have many of the same similar points, resulting in the computer which cannot be directly extracted and so on. In addition to these several cases, some special circumstances can affect as well as hinder the results of the matching algorithm and will directly affect the effect of three-dimensional modeling of the face, but also the effect of face recognition, for the function of face recognition using binocular stereo vision and depth-sensing detection algorithm for face recognition will be faster, respectively, and the efficiency of the traditional algorithm for comparison; its efficiency comparison chart is shown in Figure7.Figure 7
Face recognition efficiency graph of depth-sensing detection and traditional algorithm.


### 5.4. Demonstration of 3D Face Modeling Based on Binocular Stereo Vision with Depth-Sensing Detection

In this experiment, a bilinear face model-based algorithm is first used to represent the rough model for subsequent model refinement operations. After that, various prior constraints are used as refinement steps, which are the feature prior constraints for feature pixel reextraction, texture constraints, and regularization constraints, respectively. The rough model is rerepresented by these constraints, which is the expression form of the refined model. Then, point- and curve-based deformation optimization is performed based on the model, and concave and convex deformation is performed on the surrounding neighborhoods of the five senses regions, and the nonfeatured regions are also deformed to achieve a 3D face refinement modeling method with more accurate details of the five senses and more closely matching the texture of each region to the actual face. Finally, different face images are reconstructed to show the results of this chapter’s refinement algorithm modeling, and the evaluation criteria are applied to compare the single-viewpoint and multiviewpoint modeling methods, and the quality of the model is compared in the absence of a priori, which verifies the irreplaceability of each a priori constraint in this chapter. Finally, the comparison and validation process of the final 3D color-textured face model reconstruction algorithm of this paper with other algorithms is shown. The objective evaluation metrics show that the color-textured face model of this paper’s algorithm has better results in terms of face modeling accuracy and refinement and has different results for the efficiency of using binocular stereo vision and depth-sensing detection algorithms versus using traditional methods; the degree of comparison is shown in Figure8.Figure 8
Modeling efficiency of binocular stereo vision and depth-sensing detection algorithms compared to using traditional methods.As an extremely challenging research topic in the field of biometric recognition, 3D face recognition has been attracting the attention of many research institutions and researchers for many years because of its high academic value and wide application prospect. In the 3D model construction, a 3D modeling method combining SFS (light-dark recovery shape) and LMM (local deformation model) is used to solve the problem that SFS has a good recovery effect on the forehead and cheek parts of the face, but not on the face organs (eyes, nose, ears, etc.), and a 3D face model with high accuracy is constructed. For the problem of the high dimensionality of feature descriptors and large computation of SIFT algorithm, an improved algorithm of SIFT feature extraction based on PCA is proposed. For a large number of feature points in face images, 128 dimensions mean a huge amount of computation. The dimensionality of the feature descriptors is improved by reducing the dimensionality of the descriptors to 48 dimensions, which reduces the computation volume of the improved algorithm compared with the original SIFT algorithm and can improve the speed of face recognition more effectively. Compared with the original SIFT algorithm, the recognition time is reduced.The improved SIFT algorithm is used to experiment with two-dimensional images and three-dimensional models in video recognition. From the experimental results, not only does it achieve a higher face recognition rate but also can effectively improve the computational efficiency and shorten the time spent on the face recognition process. The experiments are also simulated with occluded objects, and the results show that 3D faces have higher recognition rates than 2D images in the presence of occluded objects and have better robustness to the occlusion situation. The three-dimensional face can better identify the feature points, and the follow-up research can identify the feature point algorithm, which can better promote the development of research.


## 5.1. Analysis of Face Feature Point Data Collection

To test the effectiveness of the algorithm, the following two steps are used to test and compare the effect of 3D face reconstruction. In the first step, the 3D face database is directly used, projected as left and right images, matched by the algorithm proposed in this paper, and compared with the actual parallax map; in the second step, the 3D face reconstruction experiment is carried out, and the 3D face reconstruction is completed by the self-designed stereo vision system. In the process of algorithm design, this paper combined the matching cost with the adaptive weighting, and in the database test, the left and right images of the 3D FRD projection of Texas were processed. The efficiency of face feature point acquisition using binocular stereo vision-based depth-sensing detection compared with the traditional method is shown in Figure5.Figure 5
Comparison of the efficiency of binocular stereo vision and depth-sensing detection with the traditional way to identify feature points.The above experiments confirm that the dense matching algorithm can indeed be applied in face matching, and the following paper needs to validate the method proposed autonomously in this paper to see if it can improve in terms of time and error. In practical application scenarios, it is not possible to know the parallax range of a specific object in advance, so a larger range is usually chosen to ensure that the corresponding match point can be found. However, this not only prolongs the search time but also makes it very easy to have a false match. However, the face features are common, and the parallax of its feature points can be explored first so that the maximum and minimum parallax can be clarified, which is helpful to get the parallax accurately in a shorter period.


## 5.2. Analysis of the Efficiency of 3D Face Modeling

This experiment is divided into 5 groups: first, the first group according to the actual maximum parallax and minimum parallax to determine the search interval, the other four groups in the matching, the maximum parallax, and minimum parallax following the gradient of 5 pixels to expand the standard, the matching time and error of these five groups for comparative analysis, each group need to identify 50 images, where the longer the matching time of binocular stereo vision and depth-sensing detection algorithm, the longer the matching time of binocular stereo vision and depth-sensing detection algorithm, the larger the search interval. The specific matching effects of the four matching algorithms were tested, and some parameters of the algorithms were adjusted. The validation results show that the proposed method has made some progress in both time and error. This section concludes by applying the matching algorithms to the face images obtained through the binocular vision system.The results confirm that the reconstruction effect is eligible, a certain number of models are selected, the worst 5 cases of each method are eliminated, 2 models with larger differences are used for each group to evaluate, and the distribution of the average normalized NME about feature points is calculated for 10 groups.The same number of 68 points is extracted in the same order. And what can be seen is that the extracted feature pixels located at the facial contour, eye boundary, eyebrow boundary, mouth, and nose boundary cover all the important parts of the human face, which are completely accurate to represent the personalized human face. Therefore, the 68 feature point coordinate positions are used as a feature prior constraints, and then, the accurate 3D face model contour estimation is performed, i.e., model refinement, global optimization, and texture colorization. The feature prior constraint is considered an accurate estimation of the face 3D facial model, but its number of features is after all limited and there is no color pixel information. Therefore, there is not enough information to refine the nonfeature regions except for the overall face model which is more obvious. In contrast, by comparing the texture differences of matching pixels between two images, the detailed information of the face can be mined in-depth, but in texture-free regions and repeated texture regions, these algorithms are often unable to reconstruct 3D face models with high precision because there is not enough visual information to be acquired. Fully mining the 3D information implied between multiview images and introducing adaptive texture constraints can improve the accuracy of the estimated 3D face models. Experiments demonstrate that the use of binocular stereo vision with depth-sensing detection algorithms has a significant effect improvement compared with traditional modeling methods, and the efficiency comparison is shown in Figure6.Figure 6
Comparison of binocular stereo vision and depth-sensing detection algorithms with conventional modeling efficiency.


## 5.3. Face Recognition Analysis

In the so-called stereo matching algorithm, that is, a point in the physical world is mapped to the projection point matching process in the left and right imaging pictures of two binocular cameras, the usual process is first a point selection in the image of one camera and then according to the coordinates of this point in the picture of the other camera to match the corresponding point search. If it is based on the polar line constraint, the search can be matched on the same line in the same vertical coordinate, which can reduce the search range greatly. However, there are some other problems and challenges beyond solving the search range only, for example, the same physical area in the photos of 2 cameras due to the different direction and intensity of lighting, and also intermingled with noise effects so that the grayscale color of the matching corresponding points of 2 different cameras at the same point is very different; it is also possible that directly due to the different angles that a point is present in one camera, there is no corresponding point in the photo of the other camera; there are also images due to the repetition of feature points so that the photos of both cameras have many of the same similar points, resulting in the computer which cannot be directly extracted and so on. In addition to these several cases, some special circumstances can affect as well as hinder the results of the matching algorithm and will directly affect the effect of three-dimensional modeling of the face, but also the effect of face recognition, for the function of face recognition using binocular stereo vision and depth-sensing detection algorithm for face recognition will be faster, respectively, and the efficiency of the traditional algorithm for comparison; its efficiency comparison chart is shown in Figure7.Figure 7
Face recognition efficiency graph of depth-sensing detection and traditional algorithm.


## 5.4. Demonstration of 3D Face Modeling Based on Binocular Stereo Vision with Depth-Sensing Detection

In this experiment, a bilinear face model-based algorithm is first used to represent the rough model for subsequent model refinement operations. After that, various prior constraints are used as refinement steps, which are the feature prior constraints for feature pixel reextraction, texture constraints, and regularization constraints, respectively. The rough model is rerepresented by these constraints, which is the expression form of the refined model. Then, point- and curve-based deformation optimization is performed based on the model, and concave and convex deformation is performed on the surrounding neighborhoods of the five senses regions, and the nonfeatured regions are also deformed to achieve a 3D face refinement modeling method with more accurate details of the five senses and more closely matching the texture of each region to the actual face. Finally, different face images are reconstructed to show the results of this chapter’s refinement algorithm modeling, and the evaluation criteria are applied to compare the single-viewpoint and multiviewpoint modeling methods, and the quality of the model is compared in the absence of a priori, which verifies the irreplaceability of each a priori constraint in this chapter. Finally, the comparison and validation process of the final 3D color-textured face model reconstruction algorithm of this paper with other algorithms is shown. The objective evaluation metrics show that the color-textured face model of this paper’s algorithm has better results in terms of face modeling accuracy and refinement and has different results for the efficiency of using binocular stereo vision and depth-sensing detection algorithms versus using traditional methods; the degree of comparison is shown in Figure8.Figure 8
Modeling efficiency of binocular stereo vision and depth-sensing detection algorithms compared to using traditional methods.As an extremely challenging research topic in the field of biometric recognition, 3D face recognition has been attracting the attention of many research institutions and researchers for many years because of its high academic value and wide application prospect. In the 3D model construction, a 3D modeling method combining SFS (light-dark recovery shape) and LMM (local deformation model) is used to solve the problem that SFS has a good recovery effect on the forehead and cheek parts of the face, but not on the face organs (eyes, nose, ears, etc.), and a 3D face model with high accuracy is constructed. For the problem of the high dimensionality of feature descriptors and large computation of SIFT algorithm, an improved algorithm of SIFT feature extraction based on PCA is proposed. For a large number of feature points in face images, 128 dimensions mean a huge amount of computation. The dimensionality of the feature descriptors is improved by reducing the dimensionality of the descriptors to 48 dimensions, which reduces the computation volume of the improved algorithm compared with the original SIFT algorithm and can improve the speed of face recognition more effectively. Compared with the original SIFT algorithm, the recognition time is reduced.The improved SIFT algorithm is used to experiment with two-dimensional images and three-dimensional models in video recognition. From the experimental results, not only does it achieve a higher face recognition rate but also can effectively improve the computational efficiency and shorten the time spent on the face recognition process. The experiments are also simulated with occluded objects, and the results show that 3D faces have higher recognition rates than 2D images in the presence of occluded objects and have better robustness to the occlusion situation. The three-dimensional face can better identify the feature points, and the follow-up research can identify the feature point algorithm, which can better promote the development of research.


## 6. Conclusion

This paper proposes a modeling method combining binocular stereo vision and depth-sensing detection algorithm, which is fast and convenient but still needs to be improved in terms of the constructed texture information, directly through the acquisition of images after the software generates models.The modeling method based on the combination of binocular stereo vision and depth-sensing detection algorithm has achieved good results in the experiment, but there are still some problems that need to be solved. This paper mainly focuses on face recognition comparison in the simulated occlusion situation; after which the comparison can be simulated in the light, expression changes, and other aspects of in-depth research, and more and more researchers are exploring and studying the 3D face feature extraction method, and each method has its advantages and disadvantages, and future research can try to combine the improved algorithm in this paper with other algorithms to further improve the face recognition rate.Using binocular stereo vision algorithm and depth-sensing detection algorithm to model 3D face, the efficiency of modeling and recognition has been greatly improved. All the experiments in this paper have proved this problem.



---

*Source: 2321511-2022-07-15.xml*