# CT-Video Matching for Retrograde Intrarenal Surgery Based on Depth Prediction and Style Transfer
## 3 Results  
This section verifies the effectiveness of the proposed methods. First, we examined the results of our depth prediction model. As shown in Table 3, we compared the effects of different loss functions on the experiment. We found that the difference in loss functions had little impact on the performance of the model. The loss functions and error functions are shown in 2.1.2, with \(\delta\)1, \(\delta\)2, and \(\delta\)3 was set as 1.25, 1.25\({}^{2}\), and 1.25\({}^{3}\), respectively. The accuracy of the model could achieve 94.1% in depth prediction on the SI-DM dataset using the loss function of L2 + SSIM. Figure 6 shows some examples of SI-DM's test dataset. We could find that the predicted depth map was generally consistent with the ground truth. Figure 5: The process of semantic feature matching. Figure 6: Examples of model depth map prediction results. The first row shows virtual endoscopic images (SI), the second row shows the predicted depth map, and the last row displays the ground truth of the depth map. Then, we evaluated the effectiveness of style transfer by analyzing the feature distributions of the source domain images (EI), the target domain images (SI), and the images generated by the CycleGAN. We input each of these three types of images into the discriminator of the target domain (Discriminator B in Figure 3, which determines whether an image belongs to the target domain). Each image was denoted by a \(1\times 1024\) feature vector. We used t-SNE (t-distributed stochastic neighbor embedding) to visualize these features in the same 2D coordinate space (see Figure 7a). We could see that the images generated by the CycleGAN were closer to those in the target domain, but due to the discriminator having a discriminatory effect, the feature distribution of both has a certain boundary. Similarly, these images were also inputted to ResNet-50 for feature vectoring and visualization. The results are shown in Figure 7b; the distribution of the images generated by the CycleGAN and target domain images are similar with some overlapping areas, which meant that the conventional CNN classifier could not distinguish between the two types of images correctly. This proved that we could achieve the migration between domain distributions through style transfer. The examples of style transfer results are shown in Figure 8.  
In addition, in order to further prove the effectiveness of the style transfer, we directly input the ureteroscopic images into the depth prediction model, and the results of the depth maps are shown in Figure 9b. For comparison, we input the same ureteroscopic images into the depth prediction model after the style transfer and got the depth maps as shown in Figure 9d. As shown in Figure 9, we can see that, by comparison, the depth \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Loss Function**} & \multicolumn{4}{c}{**Error\(\downarrow\)**} & \multicolumn{4}{c}{**Threshold Accuracy\(\uparrow\)**} \\ \cline{2-7}  & **RMSE** & **RMSE (log)** & **log10** & **Rel** & **\(\mathbf{\xi}\)1** & **\(\mathbf{\xi}\)2** & **\(\mathbf{\xi}\)3** \\ \hline L1 & **0.088** & **0.223** & **0.197** & **0.256** & **0.730** & 0.870 & 0.933 \\ \hline L2 & 0.095 & 0.235 & 0.208 & 0.269 & 0.701 & 0.856 & 0.928 \\ \hline SSIM & 0.102 & 0.258 & 0.227 & 0.304 & 0.675 & 0.844 & 0.917 \\ \hline L1 + SSIM & 0.098 & 0.239 & 0.210 & 0.271 & 0.689 & 0.869 & 0.936 \\ \hline L2 + SSIM & 0.091 & 0.223 & 0.198 & **0.256** & 0.721 & **0.881** & **0.941** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of model depth prediction results under different loss functions. Figure 7: Feature distribution results. (**a**) represents the feature distribution results of the images input into the discriminator (D) network, and (**b**) displays the feature distribution results of the images input into the ResNet-50 network. Since D has the discriminant effect, while ResNet-50 does not have, so b_real and b_fake are closer in the feature distribution.