## 4 Classifier-Independent Level of Separability Measures
### Ordinal-Scaled and Categorical Features  
For ordinal-scaled features, we propose to map the features to the set \(\{1,\ldots,n_{i}\}\), whereby \(n_{i}\) denotes the number of possible values of feature \(i\). For instance, let us assume that feature \(i\) consists of the three feature values _low_\(<\)_medium_\(<\)_high_. Then, the values of feature \(i\) are transferred to the set \(\{1,2,3\}\), i.e., mapping _low_ to \(1\), _medium_ to \(2\), and _high_ to \(3\)  
Figure 5: Visualisation of the discriminant ratio components for two 2-dimensional classes. For categorical features, we propose to replace the mean value by the _mode_, i.e., the most frequent value. Let \(D\) be a \(d\)-dimensional categorical feature space. By \(\Delta:D\times D\rightarrow\{0,1\}^{d}\), we denote the \(d\)-dimensional _difference vector_, which we define as follows. Let \(x,z\in D\) be two data points from \(D\). Furthermore, let \(\delta\in\{0,1\}^{d}\) be the difference between \(x\) and \(z\), i.e., \(\delta=\Delta(x,z)\). Then, for all \(i=1,\ldots,d\), the components of \(\delta\) are defined as follows:  
\[\delta_{i}= \begin{cases}0,&x_{i}\
eq z_{i},\\ 1,&x_{i}=z_{i}.\end{cases}  \tag{13}\]  
Note that in general, the mode is not unique. Thus, by \(\bar{M}_{i}\), we denote the set of all \(d\)-dimensional mode values specific to \(X_{\Omega}^{i}\), i.e.,  
\[\bar{M}_{i}=\Big{\{}\text{mode}(X_{\Omega}^{i})\Big{\}}. \tag{14}\]  
Therefore, for the categorical case, with \(N_{i}=|X_{\Omega}^{i}|\), we define the \(d\)-dimensional variance vector, as follows:  
\[\sigma_{i}^{\text{cat}}=\frac{1}{(N_{i}-1)\cdot|\bar{M}_{i}|}\sum_{x\in X_{ \Omega}^{i}}\sum_{x_{\text{cat}}\in\bar{M}_{i}}\Delta(x,\bar{x}_{\text{cat}}). \tag{15}\]  
Note that it is not necessary to apply the Hadamard product in the case of categorical features for the following reason: The elements of the difference vector are always equal to zero or one. Thus, squaring does not change the vector elements. Analogously to the non-categorical case, we define the _categorical discriminant ratio_, denoted by \(\text{DR}^{\text{cat}}\), as follows:  
\[DR^{\text{cat}}_{i,j}:=\frac{\left\|\frac{1}{|\bar{M}_{i}|+|\bar{M}_{j}|}\sum_ {\text{cat}\in\bar{M}_{i}}\sum_{\text{cat}\in\bar{M}_{j}}\Delta(\bar{x}_{ \text{cat}},\bar{z}_{\text{cat}})\right\|^{2}}{\|\sigma_{i}^{\text{cat}}\|+\| \sigma_{j}^{\text{cat}}\|}. \tag{16}\]  
As an example, we constructed a non-numerical data set that is depicted in Table 2, based on the authors' meta information. From Table 2, we obtain the following unique 3-dimensional mode value, based on the features Middle Name, Institute, and ORCID,  
\[\bar{x}_{\text{cat}}=(\text{No},\text{MSB},\text{Yes}).\]  
In combination with the obtained mode value, \(\bar{x}_{\text{cat}}\), we can compute the categorical variance as follows, based on Equation (15),  
\[\sigma^{\text{cat}}=\frac{1}{3-1}\sum_{i=1}^{3}\Delta(x_{i},\bar{x}_{\text{cat }})=\frac{1}{2}\left[ \begin{pmatrix}0\\ 0\\ 1\end{pmatrix} + \begin{pmatrix}1\\ 0\end{pmatrix} + \begin{pmatrix}0\\ 1\\ 0\end{pmatrix} \right]=\frac{1}{2} \begin{pmatrix}1\\ 1\\ 1\end{pmatrix} .\]  
If we include the first author of this work to the current data set, we get the additional feature vector \(x_{4}=(\text{No},\text{NIP},\text{Yes})\). This leads then to two different \(\bar{x}_{\text{cat}}\) values, i.e., \(\bar{x}_{\text{cat}}\in\{(\text{No},\text{MSB},\text{Yes}),(\text{No},\text{ NIP},\text{Yes})\}\). Note that in this example, all of the three (four) data points belong to one class, therefore we omitted the index in \(\sigma^{\text{cat}}\). \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Author** & **Middle Name** & **Institute** & **ORCID** & **Notation** \\ \hline [PERSON] & No & MSB & No & \(x_{1}\) \\ [PERSON] & Yes & MSB & Yes & \(x_{2}\) \\ [PERSON] & No & NIP & Yes & \(x_{3}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Example Data Set. MSB: Medical Systems Biology. NIP: Neural Information Processing.