# A New Approach to Border Irregularity Assessment with Application in Skin Pathology
## 3 Results
### Evaluation Metrics  
To evaluate our method we used both quantitative and qualitative measures. The aim of the quantitative analysis was to answer the two following questions: (1) what was the proportion of retes detected in the correct location?, and (2) how severe is the problem of overdetection (i.e., false positives)? The qualitative analysis helps to assess how closely the morphometry of automatically segmented retes match the morphometry of manually segmented ones. Figure 21: Determining the projection’s morphometry: (**a**) the projection’s length is approximately the maximum value of the geodesic distance transform of the projection’s region with seeds located at the projection’s base, (**b**) the projection’s height is the maximum distance of a boundary point \(P\) from the straight line \(k:Ax+By+C=0\) containing the projection’s base. Both manually and automatically segmented retes bases are actually segments in the same 2D image space. Since it is virtually impossible for any pair of manually and automatically segmented retes to perfectly overlap, as typically the corresponding endpoints would be located at least a few pixels away, we defined the following matching criterion: in order to consider a rete \(R_{1}\) as matching a rete \(R_{2}\) the distance between the centers of their bases must be at most \(l_{2}/2\), where \(l_{2}\) is the length of \(R_{2}\)'s base. Let us define the set of automatically segmented retes as \(\mathbf{R}_{\text{A}}\) and the set of manually segmented retes as \(\mathbf{R}_{\text{M}}\). If either: (1) \(R_{\text{A}}^{i}\in\mathbf{R}_{\text{A}}\) is matched by exactly one \(R_{\text{M}}^{j}\in\mathbf{R}_{\text{M}}\) and \(R_{\text{M}}^{j}\) is matched only by \(R_{\text{A}}^{i}\), or (2) \(R_{\text{A}}^{i}\in\mathbf{R}_{\text{A}}\) is matched by exactly one \(R_{\text{M}}^{j}\in\mathbf{R}_{\text{M}}\) and \(R_{\text{M}}^{j}\) is not matched with any rete from \(\mathbf{R}_{\text{A}}\), or (3) \(R_{\text{M}}^{j}\in\mathbf{R}_{\text{M}}\) is matched by exactly one \(R_{\text{A}}^{i}\in\mathbf{R}_{\text{A}}\) and \(R_{\text{A}}^{i}\) is not matched with any rete from \(\mathbf{R}_{\text{M}}\), then we say that these two retes are matched exclusively. If a given manually segmented rete is matched by multiple automatically segmented retes, then those automatically segmented retes should be merged. On the other hand, if a given automatically segmented rete is matched by multiple manually segmented retes, then that automatically segmented rete should be split. To measure the performance of our method we used the Jaccard index. The Jaccard index is a statistic used for comparing the similarity and diversity of finite sample sets, and is defined as the size of the intersection divided by the size of the union of the sample sets:  
\[J(A,B)= \begin{cases}\frac{|A\cap B|}{|A\cup B|}=\frac{|A\cap B|}{|A|+|B|-|A \cap B|}&\text{if }A\cup B\
eq\emptyset\\ 1&\text{otherwise}\end{cases}  \tag{12}\]  
In our case the two sets are a set of manually segmented retes--the ground truth (denoted as MAN)--and a set of automatically segmented retes (AUT). We computed two metrics based on the Jaccard index. In the restrictive one, the intersection set consists only of retes matching exclusively:  
\[J_{\text{strict}}=\frac{\text{\# true positive MANs}}{\text{\# MANs}+\text{\# false positive AUTs}} \tag{13}\]  
In the relaxed one, non-exclusively matching automatically segmented retes (i.e., both retes to be merged and to be split) also belong to the intersection set:  
\[J_{\text{relaxed}}=\frac{\text{\# MANs}-\text{\# false negative MANs}}{\text{\# MANs}+\text{\# false positive AUTs}} \tag{14}\]  
We included the relaxed version as in many cases the decision made by a pathologists whether to treat a projection in epidermis as one rete with two tips or two separate retes is arbitrary (Figure 22). In a perfect case, when all ground truth retes are matched and there are no automatically segmented retes without a match, the Jaccard index equals 1. Otherwise its value is in the range \([0,1)\).