## 3 System Building Blocks
### Modeling
#### 3.2.2 Filtering  
The next step in the system pipeline is the filtering, which is necessary to smooth the noisy 3D reconstruction. We thus developed a specific filter for face meshes: a content-aware filter for RGB-D faces that proposes the smoothness of each point of a given mesh by comparing the local neighborhood using a set of exemplars [15]. This filter consists of an exemplar-based neighborhood matching, where all models are in a frontal position, avoiding rotation and perspective. We take advantage of prior knowledge of the models (faces) to improve the comparison. We first detect facial feature points, create the point correctors for regions of each feature and only use the corresponding regions to correct a point of the filtered mesh. As a result, the model is invariant to depth translation and scale. The proposed method is evaluated on a public 3D face dataset with different levels of noise. The results show that the method is able to remove noise without smoothing the sharp features of the face. Figure 7 illustrates the proposed filtering method used [15]. The filtering model comprises two main steps: the model standardization and the filtering itself. The goal of the model standardization is to allow different scales and sampling frequencies to be handled due to the distinct acquisition processes--notice that here we propose a system that can handle different types of cameras. The model standardization process consists of changing the frequency sampling of a given model, named target \(\Pi\), according to a base model, named exemplar \(\Omega\). In cases when we use more than one exemplar to define the filter, one of them is chosen to be the base, and the others are also resampled. Firstly, we use a set of facial feature points, also known as facial landmarks [31] (see Figure 6, left), to align and resample the faces. After defining the landmarks of the two models to be aligned, an Iterative Close Point (ICP) [32] algorithm is then computed. This method returns a scale \(s\in\mathbb{R}\), a rotation \(R(2\times 2\) matrix) and a translation \(c\in\mathbb{R}^{2}\) that when applied over the second model aligns it with the first model, thus minimizing the difference between the two point sets. It is an affine transformation that can be represented in homogeneous coordinates by the following matrix:  
\[\Gamma=\left[ \begin{array}{c|c}sR&\begin{array}{c}c\cdot x\\ c\cdot y\\ \hline 0&0\end{array}&\begin{array}{c}1\\ \end{array}\end{array} \right]_{3\times 3} \tag{2}\]  
After the alignment, we can perform the target resampling process. This consists of creating a rectangular grid of target points, named  
\(\Delta=\{(x_{kl},y_{kl},z_{kl},u_{kl},v_{kl});k=1\dots M,l=1\dots N\}\). It is performed by the definition of the coordinates \(XYZ\) and \(UV\) of \(M\times N\) points (\(\Delta\) dimension) taken regularly into the target texture space. The resampling is based on a triangulation of points of \(\Pi\) in UV space. The definition of \(M\) and \(N\) (\(\Delta\) dimension) is based on the target \(FF\)s transformed into the exemplar texture space. Each target \(FFP\) (into the target texture space) is multiplied on homogeneous coordinates by \(\Gamma\). An oriented bounding box is created around these transformed points. Finally, \(M\) and \(N\) are the dimensions of this box. Once these dimensions are defined, it is necessary to create the \(M\times N\) points of \(\Delta\). The resampling starts with texture (samples inside the face bounding box in target texture space), and for each sample, we need to define the respective XYZ and UV coordinates. It is also necessary to define \((j,i)\) for each sample into target texture space and then the respective \((u,v)=\psi^{-1}(j,i)\). The coordinates \((x,y,z)\) are obtained based on this \((u,v)\). We create a Delaunay triangulation of the UV coordinates of all points of \(\Pi\) and detect the triangle composed of \(p_{a}\), \(p_{b}\) and \(p_{c}\) that contains \((u,v)\).