# Realistic Texture Mapping of 3D Medical Models Using RGBD Camera for Mixed Reality Applications
## 2 Materials and Methods
### Registration of DICOM-Based and RGBD Pointclouds
#### 2.3.2 Minimizing Distance between Contours  
To further improve the registration result between the two pointclouds, an algorithm based on aligning their contours has been developed. The algorithm is based on the iterative application of different steps:  
1. Projection on a rotated plane: the reference system \(O_{xyz}^{RGDD}\) was rotated according to the random rotation vector \(R_{1}=(\alpha,\beta,\gamma)\) defining the reference system \(O_{xyz_{\alpha\beta}}^{RGD}\). In detail: \(\alpha\) defines a rotation on the \(xz\) plane, \(\beta\) defines a rotation on the \(yz\) plane, and \(\gamma\) defines a rotation on the \(xy\) plane. Then, RGBD and DICOM-based pointclouds were projected onto the \(xy\) plane of the \(O_{xyz_{\alpha\beta}}^{RGBD}\) reference system, defining two 2D images. Lastly, the objects' contours were identified by evaluating the concave hull of the images, defining the 2D points vectors \(c_{RGB}\) and \(c_{DICOM}\). \(R_{1}\) vector random values were chosen in the range \(\pm 0.5236\) radians (i.e., \(\pm 30^{\circ}\)). 2. Application of the distance transform filter: applying the distance transform filter to the contour \(c_{DICOM}\), the distance matrix was created. A distance transform filter is a mathematical operation that evaluates the distance of each pixel in an image to the nearest boundary or feature in the image. The result is a new image called the distance map or distance transform map, where each pixel value represents the distance between that pixel and the nearest object boundary. Specifically, the distance transform filter implemented in the OpenCV library was used and an example of its application on a \(c_{DICOM}\) contour is shown in Figure 9c. 3. Evaluation of optimized rotation vector \(R_{2}\): the distance between contours \(c_{RGB}\) and \(c_{DICOM}\) was minimized based on the distance matrix image. Specifically, the Nelder-Mead method [46] minimization algorithm, implemented in the OpenCV library, was used. The parameters used for the minimization are: \(\theta\) (rotation on the \(xy\) plane), \(t_{x}\) (translation along the \(x\) axis), and \(t_{y}\) (translation along the \(y\) axis). At the end of the minimization process, the optimized parameters vector was defined: \(R_{2}=(\theta_{opt},t_{xopt},t_{yopt})\). Lastly, optimal translations (\(t_{xopt}\), \(t_{yopt}\)) were converted from volume-based translations to millimeter-based translations using the slice thickness parameter (Equation (1)). 4. Application of \(R_{2}\) vector: the RGBD pointcloud was then permanently rotated and translated using values evaluated at point (3). 5. Stopping criterion: a total of one thousand random points are identified on the RGBD pointcloud and the average euclidean distance was calculated with respect to the DICOM-based pointcloud. At the end of this procedure, if the mean registration error \(E\) was greater than a threshold value \(E_{thr}\), the entire registration process was repeated by defining a new random rotation vector \(R_{1}\); otherwise, the registration process was terminated. In particular, a threshold value \(E_{thr}=1\) mm was used. For better clarification, the highlights of this method are shown in Figure 9 and the flowchart diagram of the algorithm is illustrated in Figure 10.  
Figure 10: Flowchart diagram of the contours-based minimization algorithm. Figure 9: Highlights of the contours-based minimization algorithm. (**a**) RGBD pointcloud contour \(c_{RGB}\) after its projection on the \(xy\) plane defined by the rotation vector \(R_{1}=(0,0,0)\); (**b**) DICOM-based pointcloud contour \(c_{RGB}\) after its projection on the \(xy\) plane defined by the rotation vector \(R_{1}=(0,0,0)\); (**c**) Application of the distance matrix filter to the \(c_{DICOM}\) contour. The colour scale from black to white means the position of a pixel closer and farther from the reference contour, respectively; (**d**) Minimization of the distance between \(c_{RGB}\) and \(c_{DICOM}\) based on the distance matrix.