# Fully Leveraging Deep Learning Methods for Constructing Retinal Fundus Photomatages
## 3 Experimental Results
### Quantitative Evaluation  
We measure quantitative performance in two terms: (1) the number and portion of successfully integrated frames for each sequence and (2) the target registration error (TRE) based on landmark points. A frame is deemed as successfully integrated into the montage unless the keypoint matching based rigid registration fails, even with the safeguard measure, which is determined by manual inspection. Landmark points for which TRE is measured are defined as vessel bifurcation points that are automatically detected from the vessel segmentation map using the PlantCV library [27]. We note that, since the registration results vary considerably for each comparative method, TRE could not be measured on the same keypoints or the same image pairs. Thus, we only provide an aggregate comparison of the average TRE. Because the number of integrated framesare compared within an identical setup, we believe that these results present a more straightforward comparison of the performance between methods. ```
Input : Set of fundus image frames \(\mathcal{F}\),  Trained Faster R-CNN for detecting optic disc and fovea  Trained SSANet for vessel segmentation Output : Constructed photomontage \(\mathcal{M}\), vessel map montage \(\mathcal{V}\) FunctionInitMontage(\(f\)):  Get \(v\) = SSANet vessel segmentation on \(f\)  Set \(\mathcal{M}\) = \(f\) and \(\mathcal{V}\)  FunctionAddToMontage(\(f\)):  Get \(v\) = SSANet vessel segmentation on \(f\)  Preprocess \(f\) using \(v\)  Get homography \(h\) from keypoint based rigid registration of \(f\) into \(\mathcal{M}\)  Get transform \(\mathcal{T}(f)=h\times f\) ifDifference in area between \(\mathcal{T}(f)\) and \(f<10\%\)then  Transform \(h\times v\) to align with \(\mathcal{V}\)  Get displacement vector map \(\mathcal{D}\) from B-spline based non-rigid registration of \(v\) into \(\mathcal{V}\)  Warp \(\mathcal{D}(v)\) and add into \(\mathcal{V}\)  Warp \(\mathcal{D}(f)\) to align with \(\mathcal{M}\)  Blend \(\mathcal{D}(f)\) with \(\mathcal{M}\) for all frames \(f_{i}\) in \(\mathcal{F}\)do  Detect optic disc and fovea in \(f_{i}\) and store center points of fovea \(p_{i}^{f}\) and optic disc \(p_{i}^{o}\) ifboth optic disc and fovea are detected in \(f_{i}\)then add \(f_{i}\) to \(\mathcal{F}^{f\&o}\) elseifonly optic disc is detected in \(f_{i}\)then add \(f_{i}\) to \(\mathcal{F}^{f}\) elseadd \(f_{i}\) to \(\mathcal{F}^{p}\)  end if  InitMontage(\(f_{init}\) = \(\arg\min_{f_{i}\in\mathcal{F}^{f\&o}}||p_{i}^{f}-p_{i}^{center}||\)), \(p_{i}^{center}\) is image center of \(f_{i}\)  Set \(p_{init}^{o}\) and \(p_{init}^{f}\) as optic disc and fovea position of \(f_{init}\) while\(\mathcal{F}^{f\&o}\) is not emptlydo  AddToMontage(\(f_{next}\) = \(\arg\min_{f_{i}\in\mathcal{F}^{f\&o}}||p_{i}^{o}-p_{init}^{o}||\)), pop \(f_{next}\) from \(\mathcal{F}^{f\&o}\)  end while\(\mathcal{F}^{o}\) is not emptlydo  AddToMontage(\(f_{next}\) = \(\arg\min_{f_{i}\in\mathcal{F}^{o}}||p_{i}^{o}-p_{init}^{o}||\)), pop \(f_{next}\) from \(\mathcal{F}^{o}\)  end whilewhile\(\mathcal{F}^{f}\) is not emptlydo  AddToMontage(\(f_{next}\) = \(\arg\min_{f_{i}\in\mathcal{F}^{f}}||p_{i}^{f}-p_{init}^{f}||\)), pop \(f_{next}\) from \(\mathcal{F}^{f}\)  end whilewhile\(\mathcal{F}^{p}\) is not emptlydo  AddToMontage(\(f_{next}\) = \(\arg\max_{f_{i}\in\mathcal{F}^{f\&o}}\) keypoint matches between \(f_{i}\) and \(\mathcal{M}\)), pop \(f_{next}\) from \(\mathcal{F}^{p}\)  end while
```  
**Algorithm 1**Retinal Fundus Photomontage Construction Using Deep Learning. Table 1 presents the quantitative evaluation. As to serve as an ablation study, we provide a comparison with the results when using the greedy approach that is based on the number of keypoint matches for sorting all frames to show the effect of our object detection based frame sorting method. We also provide a comparison between the results  
for other preprocessing methods, including a simple approach to normalize the minimum and maximum pixel intensities, and a more complex modified morphological top-hat transform by [PERSON] et al.