# Towards Understanding Clustering Problems and Algorithms: An Instance Space Analysis
## 5 Plotting Real Datasets in Instance Space
### Generating an Instance Space from Combined Artificial and Real Datasets  
A new instance space was generated from a total of 599 datasets, where 380 are the same artificial datasets used in Section 3 and another 219 real datasets were collected from the OpenML repository and used as benchmark for clustering algorithms in [11]. In these datasets, the number of examples ranges from 100 to 5000, the number of attributes ranges from 2 to 168, and the number of embedded clusters ranges from 2 to 30. One must notice that these datasets are representatives of classification problems. In classification problems, the objective is to find a decision model able to partition the data items into some previously defined categories. The datasets are then labeled and the model is trained to somewhat reproduce such labels. A same class can have data points spanning different regions of the input space. This differs from clustering, in which there are no pre-defined categories and the groups are usually based on their proximity. Whilst a problem for which the instances are distributed uniformly all over the input space with a clear linear decision border between the classes can be considered quite easy for classification, this is not the case for clustering (in fact, there is no cluster structure for data points randomly distributed in the input space). Therefore, although the usage of classification datasets in the evaluation of clustering algorithms is largely employed, some care must be taken in the interpretation of the obtained results. The ground truths of the real datasets were not considered in the analysis. The same procedure used with artificial datasets was employed: the clusters were defined by the Silhouette method, using the implementation of the NbClust package from R. The categorization of artificial datasets described in the previous section has remained unchanged (namely ellipsoidal, Gaussian and multiple shapes). Equation (6) presents the new projection matrix. We can see that 10 meta-features were selected. A total of six remained in the projection matrix after the inclusion of the real datasets: kurtosis, perc_out, ratio_pca, avg_abs_cor, low_dist and avg_nnd. This indicates that these measures have a relevant influence in the characterization of both types of datasets. \[\left[ \begin{array}{c}Z_{1}\\ Z_{2}\end{array} \right]=\left[ \begin{array}{cc}0.110&-0.249\\ -0.084&-0.195\\ -0.201&0.094\\ -0.270&-0.157\\ 0.071&0.287\\ 0.178&0.224\\ 0.197&0.019\\ -0.197&0.013\\ 0.163&0.023\\ 0.077&-0.132\\ \end{array} \right]^{\top}\left[ \begin{array}{c}\text{number\_ftr}\\ \text{kurtosis}\\ \text{multi\_norm}\\ \text{perc\_out}\\ \text{ratio\_pca}\\ \text{avg\_abs\_cor}\\ \text{low\_dist}\\ \text{contrast}\\ \text{hub\_score}\\ \text{avg\_nnd}\end{array} \right] \tag{6}\]Figure 8 shows the characteristics of the datasets projected in the instance space. The number of clusters for the real datasets correspond to their number of classes. Figures 9 and 10 show the distribution of selected meta-features and algorithms footprints in the instance space, respectively. Figure 8 reveals the region of the instance space where the real datasets were plotted. They are concentrated in the central and upper left region of the instance space, partially overlapping the less complex Gaussian datasets and the multiple shapes datasets. They also fill a central and lower left area not occupied by artificial datasets, which is characterized by a reduced number of examples and intermediate number of attributes and clusters. With very few exceptions, we can verify that, in general, they are not located in the areas occupied by the ellipsoidal and the G2 datasets, that is, the most complex ones. This finding highlights the importance that artificial datasets can have in experiments and that artificial and real datasets can act as complementary instances, providing a more comprehensive analysis of clustering problems.