Spatiotemporal Correlation-Based Accurate 3D Face Imaging Using Speckle Projection and Real-Time Improvement  
Wei Xiong  
College of Computer Science, Sichuan University, Chengdu 610064, China; [EMAIL_ADDRESS] (W.X. ); [EMAIL_ADDRESS] (H.Y. ); [EMAIL_ADDRESS] (K.F. ); [EMAIL_ADDRESS] (J.Z)  
Hongyu Yang  
College of Computer Science, Sichuan University, Chengdu 610064, China; [EMAIL_ADDRESS] (W.X. ); [EMAIL_ADDRESS] (H.Y. ); [EMAIL_ADDRESS] (K.F. ); [EMAIL_ADDRESS] (J.  
light-encoding methods: fringe projection profilometry (FPP) [15] and speckle projection profilometry (SPP) [16]. In the FPP, the projector projects a series of fringe-coded patterns onto the measurement target. The fringe image modulated by the measured object is synchronously captured by a stereo camera and is processed using various phase recovery techniques, such as phase-shift profilometry [17], to obtain absolute phase information. Using these methods, it is necessary to perform phase unwrapping to eliminate phase ambiguity and convert the wrapped phase to the absolute phase. In SPP, a variety of structured light-pattern encoding strategies (e.g., non-formal codification [16] and De Bruijn sequences [18]) have been proposed to increase or enrich the surface texture information of the measured object, which can avoid the matching ambiguity problem caused by binocular stereo vision [19; 20]. Therefore, the reliability and accuracy of 3D object reconstruction results can be improved. Among these, research on speckle encoding structured light has been very extensive. Compared with sinusoidal FPP in measurement technology, the 3D measurement system based on SPP does not require the harsh conditions that are required by the precise phase shift, and only a low-cost speckle-pattern imaging system is required to obtain excellent performance of 3D reconstruction results [21]. Many companies have introduced 3D sensors based on diffractive optical element (DOE) [22] laser illumination, such as RealSense [23] and Kinect V1 [24], which have the characteristics of small size, low power consumption, and high integration. [PERSON] et al. [25] used a semi-global matching (SGM) algorithm with DOE projection to improve the accuracy of stereo matching and achieved dense dynamic scene reconstruction. However, owing to the limited number of projection speckles, the spatial resolution and measurement accuracy of the 3D reconstruction data are relatively low, making them unsuitable for special applications, such as 3D face recognition. Therefore, there is a developing trend towards projecting high-density speckle encoding patterns in order to improve the precision of 3D reconstruction. In many applications, to reduce the motion blur caused by the movement of the measured object, only one speckle-coded image is projected onto the target to reduce the acquisition time. [PERSON] et al. [26] designed a system that adaptively changed the speckle size to the optimal size required by the algorithm, according to the distance of the object. Various pre-processing methods were used to reduce the spread of speckles in order to increase the density of the 3D point cloud. [PERSON] et al. [27] presented an automatic and rapid whole-body 3D measurement system based on multi-node 3D sensors using speckle projection. [PERSON] et al. [28] proposed a single-shot 3D shape-measurement method using an end-to-end stereo-matching network with speckle projection. A high-precision absolute phase map was obtained by combining a phase-shift profile measurement and a time-phase unwrapping technology in order to construct a high-quality disparity map for training the network model, and the disparity image was inferred and predicted by the speckle matching network. [PERSON] et al. [29] proposed a high-precision 3D surface profile measurement scheme that projected only a single-shot colour binary speckle pattern and a spatiotemporal correlation matching algorithm, which was applied to the measurement of dynamic and static objects.