## 3 Tree-Based Relative Support Distance
### Relative Support Distance  
Support vectors (SVs) play a substantial role in determining the decision hyperplane in contrast to non-SV data points. We extract data points in the training data that are most likely to be the support vectors, constructing a set of support vector candidates. Given two distinct adjacent regions \(L_{1}\) and \(L_{2}\) from the previous step, let us assume the majority class label of \(L_{1}\) is \(y=1\) and that of \(L_{2}\), \(y=2\) without loss of generality. First, we calculate the centroid (\(\mathbf{m}_{c}\)) for each majority class label as follows: for an index set \(S_{c}=\{i|\mathbf{x}_{i}\in L_{c}\) and the label of \(\mathbf{x}_{i}=c\}\),  
\[\mathbf{m}_{c}=\frac{1}{n_{c}}\sum_{i\in S_{c}}\mathbf{x}_{i}, \tag{8}\]  
where \(c\in\{1,\ 2\}\) and \(n_{c}\) is the cardinality of index set \(S_{c}\). In other words, \(\mathbf{m}_{c}\) is the majority-class centroid of data points in \(L_{c}\), of which the labels are \(y=c\). Next, we create a virtual hyperplane that bisects the line from \(\mathbf{m}_{1}\) to \(\mathbf{m}_{2}\):  
\[ \begin{array}{c}\mathbf{M}=\frac{1}{2}(\mathbf{m}_{1}+\mathbf{m}_{2}),\\ \mathbf{W}=\mathbf{m}_{1}-\mathbf{m}_{2},\end{array}  \tag{9}\]  
where \(M\) is the middle point of the two majority-class centroids. The virtual hyperplane is given by \(H(\mathbf{x})=0\), where  
\[H(\mathbf{x})=\mathbf{W}^{t}(\mathbf{x}-\mathbf{M}). \tag{10}\]  
Lastly, we calculate the distance \(r_{x}\) between each data point \(x\) in \(S_{c}\) and \(\mathbf{m}_{c}\) and the distance \(h\) between each data point in \(S_{c}\) and the virtual hyperplane \(H(\mathbf{x})=0\):  
\[ \begin{array}{c}r_{\mathbf{x}_{c,l}}=\left\|\mathbf{x}_{c,l}-\mathbf{m}_{c}\right\|,\\ h_{\mathbf{x}_{c,l}}=\frac{\left|\mu(\mathbf{x}_{c,l})\right|}{\left\|\mathbf{W}\right\|}= \frac{\mathbf{W}^{t}(\mathbf{x}_{c,l}-\mathbf{M})}{\left\|\mathbf{m}_{1}-\mathbf{m}_{2}\right\|}, \end{array}  \tag{11}\]  
where \(\mathbf{x}_{c,l}\) is the \(l\)th data point belonging to \(S_{c}\). Figure 3 shows a conceptual description of \(r\) and \(h\) using the virtual hyperplane in a leaf. After calculating \(r_{x}\) and \(h_{\mathbf{x}}\), we apply feature scaling to bring all values into the range between \(0\) and \(1\). Our observation is that data points lying close to the virtual hyperplane are likely to be support vectors. In addition, data points lying close to the centroid are less likely to be support vectors. In light of these observations, we select data points lying near the hyperplane and far away from the centroid. For this purpose, we define the relative support distance \(T(r_{x},h_{\mathbf{x}})\) as follows:  
\[T(r_{x},h_{\mathbf{x}})=\frac{1}{(1+e^{-r_{x}})h_{\mathbf{x}}}. \tag{12}\]The larger \(T(r_{\mathbf{x}},h_{\mathbf{x}})\) becomes, the more likely that the associated \(\mathbf{x}\) is a support vector. The relationship between support vectors and distances \(r\) and \(h\) is illustrated in Figure 4. We use leaves \(L_{4}\) with distinct adjacent regions \(L_{1}\) and \(L_{2}\) in Figure 1. In Figure 4, the observations marked by circles are non-support vectors while those by triangles (in red) are support vectors selected after training all data by SVMs. The distances \(r\) and \(h\) of \(L_{4}\) relative to \(L_{1}\) are in Figure 4a, and the relative support distance measures in Figure 4c. Likewise, those of \(L_{4}\) relative to \(L_{2}\) are in Figure 4b,d. We observe that the observations, marked by triangles and surrounded by a red ellipsoid in Figure 1, correspond to the support vectors surrounded by a red ellipsoid in region \(L_{4}\) in Figure 4a, and they have larger values of relative support distance as shown in Figure 4c. Similarly, we notice that the observations, marked by triangles and surrounded by a green ellipsoid in Figure 4b, correspond to the support vectors surrounded by a green ellipsoid in region \(L_{4}\) in Figure 1, and they also have larger values of relative support distance as shown in Figure 4d.