## 4 Comparison Results
### Interpreting Local Metrics on a Synthetic Dataset  
We start by explaining the local metrics proposed in Section 3.1 using the simple synthetic blobs dataset, which is, as explained earlier, straightforward to classify. As such, we only studied its decision maps for the simple logistic regression classifier (we obtained very similar results for the other three considered classifiers). The blobs dataset is split into 1000 training and 500 testing samples. All three map methods--DBM, SDBM, and DV--achieved 100% on the test set for all data-wise global metrics. We then used the blobs dataset to establish and validate our _expectations_ for a 'good' decision map based on the local metrics introduced in Section 3.2 as follows:  
1. Distance to the nearest data \(N\): We expected this distance to be small for pixels close to the actual projections of data points and larger for pixels far away from these points. In other words, we expected the 2D distance (to the projections of the data points) to mimic the nD distance (to the actual data points). 2. Distance to the decision boundary \(B\): Similar to the above, we expected that the points close to the decision boundaries (in 2D) would also be close to the decision boundaries (in nD), and vice versa. 3. Class stability map \(S\): Ideally, we would have liked most pixels to have high stability values, especially those close to the decision boundaries (where we were most interested in studying in a decision map). 4. Gradient map \(G\): We expected a _smooth_ gradient map without any discontinuities or peaks. Ideally, we also would have liked low gradients close to the decision boundaries (for the same reason mentioned above for class stability). With these expectations in mind, we analyzed the results shown in Figure 4 for the three decision-map methods and the above-mentioned four local metrics. The first row in Figure 4 shows the decision maps and projected data points for the three methods. Figure 3: Aggregated global metrics for each decision map, classifier, and dataset combination. \(ACC_{C}\), \(ACC_{M}\), \(Cons_{d}\), \(Cons_{p}\), \(S\), and \(1-G\) are the global metrics defined in Section 3.1. Red dashes show the highest values across each dataset (row). The top-left numbers in each plot show the average values of \(ACC_{C}\)\({}^{T}\), \(ACC_{M}\)\({}^{T}\), \(Cons_{d}\)\({}^{T}\), \(Cons_{p}\), and \(\hat{S}\). Bold values indicate the highest value among the dataset (row). At a glance, all maps appeared to be similar. However, a deeper analysis of the local metrics revealed more details. The results for \(N\) (second row) and \(B\) (third row) met our expectations across all methods. Some discontinuities in DV, however, were noticeable--the \(N\) and \(S\) images appear to contain some 'cuts' that perturbed their overall smoothness. \(S\) (fourth row) primarily manifested as expected, with all low-value pixels situated around the decision boundaries. A notable difference arose with the SDBM, which exhibited larger regions of unstable pixels, indicating its vulnerability to projection errors. Regarding \(G\), the DBM and SDBM were smooth, whereas DV exhibited peaks, thus not satisfying our first expectation regarding \(G\). Even more interestingly, the local maxima regarding \(G\) for DV took the form of _lines_ roughly connecting the clusters of the projected points. As for our second expectation, we observed a surprising result: the DBM and SDBM exhibited relatively higher values close to the decision boundaries, whereas DV exhibited lower values in these areas. Consequently, no method simultaneously satisfied both expectations of \(G\). In conclusion, on the synthetic blob dataset, all three methods generally aligned with our expectations of the local metrics. However, despite the decision maps' similarities in this simple example, there were subtle but significant differences between them. This is the  
Figure 4: Decision maps and local metrics of the synthetic blob dataset (see Section 4.2).