# Coupling RetinaFace and Depth Information to Filter False Positives
## 2 Materials and Methods
### Depth Map Alignment and Segmentation  
Depth maps and the raw color images are segmented in tandem, similar to the method in [24]. The procedure involves transforming each sample into a six-dimensional vector and then clustering the point set using the mean shift algorithm [25]. Transformation into a six-dimensional vector is accomplished as follows. Samples in the Kinetic depth map are 3D points: \(p_{i},\ i=1,\ldots,N\), where \(N\) is the number of points. As described in [23], color cameras and the joint calibration of the depth facilitates reprojecting the depth samples. This involves mapping them onto the corresponding pixels in the color image. This process allows each point to be connected to its 3D spatial coordinates (x, y, and z), as well as the RGB color values. However, these representations cannot be directly compared because they exist in completely different spaces. In order to use the mean shift clustering algorithm to extract multidimensional vectors, it is necessary to make all components comparable. To make the color values comparable, they are converted to the CIELAB uniform color space, so colors are represented in a 3D space with lightness (L) values ranging from black (0) to white (100), (a) from green (-) to red (+), and (b) from blue (-) to yellow (+). The purpose of this conversion is to use the Euclidean distance between the color vectors in the mean shift algorithm. More formally, the algorithm works by considering each scene point's color information in the CIELAB color space, \(c\), as a 3D vector:  
\[p_{i}^{c}= \begin{bmatrix}\text{L}(p_{i})\\ \text{a}(p_{i})\\ \text{b}(p_{i})\end{bmatrix} ,\ \ \ \ i=1,\ldots,N. \tag{1}\]  
Figure 1: A general illustration of the proposed face detection approach. First, the input image undergoes a first round of detection RetinaFace (**top**). Second, segmentation is performed using the depth map of the image (**bottom**). Finally, the candidate faces undergo a filtering process to select regions with faces (**right**). The geometry, \(g\), is defined by each point 3D coordinates as:  
\[p_{i}^{g}= \begin{bmatrix}\mathrm{x}(p_{i})\\ \mathrm{y}(p_{i})\\ \mathrm{z}(p_{i})\end{bmatrix} ,\ \ \ \ i=1,\ldots,N. \tag{2}\]  
In order to ensure that the scene segmentation algorithm is not affected by the relative scaling of the point-cloud geometry and to ensure that the color distances and geometry are consistent, all components of \(p_{i}^{g}\) are normalized based on the average standard deviation of the coordinates of the point in three dimensions \(\sigma_{g}=\left(\sigma_{x}+\sigma_{y}+\sigma_{z}\right)/3\), which results in the following vector:  
\[ \begin{bmatrix}\overline{\mathrm{x}}(p_{i})\\ \overline{\mathrm{y}}(p_{i})\\ \overline{\mathrm{z}}(p_{i})\end{bmatrix} =\frac{3}{\sigma_{x}+\sigma_{y}+ \sigma_{z}} \begin{bmatrix}\mathrm{x}(p_{i})\\ \mathrm{y}(p_{i})\\ \mathrm{z}(p_{i})\end{bmatrix} =\frac{1}{\sigma_{g}} \begin{bmatrix}\mathrm{x}( p_{i})\\ \mathrm{y}(p_{i})\\ \mathrm{z}(p_{i})\end{bmatrix}  \tag{3}\]  
After normalizing the color information vectors, the final color representation is obtained by taking the average of the standard deviations of the L, a, and b color components:  
\[ \begin{bmatrix}\overline{\mathrm{L}}(p_{i})\\ \overline{\mathrm{a}}(p_{i})\\ \overline{\mathrm{b}}(p_{i})\end{bmatrix} =\frac{3}{\sigma_{L}+\sigma_{a}+ \sigma_{b}} \begin{bmatrix}\mathrm{L}(p_{i})\\ \mathrm{a}(p_{i})\\ \mathrm{b}(p_{i})\end{bmatrix} =\frac{1}{\sigma_{c}} \begin{bmatrix}\mathrm{L}( p_{i})\\ \mathrm{a}(p_{i})\\ \mathrm{b}(p_{i})\end{bmatrix}  \tag{4}\]  
After the normalization of the vectors of the color information and geometry, they are combined to produce the final representation \(f\):  
\[p_{i}^{f}= \begin{bmatrix}\overline{\mathrm{L}}(p_{i})\\ \overline{\mathrm{a}}(p_{i})\\ \overline{\mathrm{b}}(p_{i})\\ \lambda_{\overline{\mathrm{\pi}}}\\ \lambda_{\overline{\mathrm{\pi}}}\end{bmatrix}  \tag{5}\]  
The relative weight of color and geometry in the final segmentation can be controlled using the parameter \(\lambda\).