# Robust and High-Fidelity 3D Face Reconstruction Using Multiple RGB-D Cameras
## 3 Materials and Methods
### Fast Two-Stage Point Cloud Filtering  
In this section, we mainly describe a fast two-stage point cloud filtering method of coarse filtering based on a distance prior and fine filtering based on image matting, to ensure that the point cloud of the reconstructed subject was clean enough. The stages are, respectively, used to remove the overall and edge noise at the discontinuity in the depth domain. The schematic diagram of the whole process is shown in Figure 4.  
Specifically, we ensure that there is no other object between the face and the camera when we capture the face with the device shown in Figure 2. Otherwise, the complete 3D face reconstruction task would not be possible. This way, the face is the closest foreground subject to the camera; so, we first extract the current minimum value of depth \(depth_{min}\) by traversing the \(Z\) coordinates of all points in the point cloud, and the point to which the minimum depth value belongs is the closest point in the face to the \(XOY\) plane of the camera coordinate system. Meanwhile, based on the assumption that the maximum front-to-back diameter of a normal human head should not exceed \(\beta\) millimeters (mm), we  
Figure 3: Overview of the face reconstruction system. Figure 2: Schematic diagram of the acquisition platform. (**a**) The acquisition platform contains three calibrated RealSense D435 cameras and a light source. (**b**) The acquisition platform with the users. perform the first coarse filtering of the reconstructed point cloud based on the following distance interval:  
\[\{P|\forall p\in P,depth_{min}\leq p_{z}\leq depth_{min}+\beta\} \tag{1}\]  
where \(P\) is the point cloud after coarse filtering for each view, \(p\) is a point in the \(P\), and \(p_{z}\) denotes the \(Z\) coordinate of \(p\). In this way, we only need a simple traversal to obtain knowledge of the face position, which can remove most of the noise in the non-face areas of the point cloud. In order to effectively remove the remaining noise, especially the edge noise at the discontinuities in the depth domain, we combine the image matting method with deep learning for the first time to guide the fine filtering completely and accurately filter this noise. In this module, an advanced work called Robust High-Resolution Video Matting (RVM) [21], which adopts MobileNetV3-Large [22] as an effective backbone, is used in our image matting tasks that have a human as the foreground. The whole architecture consists of an encoder that extracts the features of an individual frame, a recurrent decoder that aggregates temporal information, and a deep guided filter module for high-resolution upsampling. RVM is designed explicitly for robust human video matting; however, it can still show excellent performance in image matting and can handle matting in real-time without additional input, which is why we chose it. To enhance the robustness of the image matting task, we combine multiple data sets, including VideoMatte240K (VM) and Adobe Image Matting (AIM), to train the network according to the training strategy in this study. The VM contains 484 \(\mathrm{4K/HD}\) video clips; in the AIM, we select only 420 images containing humans for training. The MobileNetV3 backbone is initialized with pretrained weights and uses a \(1\times 10^{-4}\) learning rate, while the rest of the network uses \(2\times 10^{-4}\). The final testing efficiency is about 10 ms. Please refer to Section 4 of the original article for specific training strategies. Its network output, which is the transparency image first proposed by [23], represents the mixing ratio of foreground and background. We know that a larger transparency value means that the foreground is more visible than the background in the current pixel. On the contrary, the background is relatively more obvious. The transition region of transparency should theoretically appear in the region of foreground and background separation.