# Automated Exploratory Clustering to Democratize Clustering Analysis
## 3 Materials and Methods
### Theoretical Definition  
The goal of AEC should be, to generate a set of multiple unique clusterings, each of which has the potential to be useful. However, on the other hand, this set should be as small as possible, since every cluster has to be investigated by a domain expert. As opposed to classical Automated Clustering, we will not search for a single clustering but a set of clusterings as the result of an AEC search. To find the \"best\" set of clusterings, we will define an _interestingness_ measure, which needs to be optimized. This interestingness could also be used as a Quality function in our abstract AutoML lifecycle. However, in the scope of this paper, we generate the clusterings in a traditional Automated Clustering fashion (optimizing a single CVI) and select the resulting sets based on all clusterings generated during the training phase. For this reason, the methods in this paper might also be thematically close to the field of diversification (see, e.g., [50]). However, as our focus lies on the problem of finding a good interestingness to directly optimize the clustering sets, future research should divert from the field focused on a posteriori analysis. In contrast to our previous conference paper [25], we streamlined the necessary functions and omit the previously used usefulness function. While this leads to many definitions being different, it does not change any significant content. The following notation is summarized in Table 1 for a quicker overview. We will denote \(\mathbb{U}\) as the set of all (investigated) clusterings and \(c\in\mathbb{U}\) as a single clustering. The power set of all clusterings will be denoted as \(\mathcal{P}(\mathbb{U})\) and a set of all investigated clusterings as \(C\in\mathcal{P}(\mathbb{U})\). **Definition 2** (Quality Function). : _The Quality function is defined as the vector of evaluation results of a single clustering:_  
\[Q(c):\mathbb{U}\rightarrow\mathbb{R}^{k},\,k\in\mathbb{N}. \tag{3}\]  
The Quality function outputs a vector of scalars, and in our example, this will be a number of different CVI evaluations for the clustering. The Quality function can be arbitrarily complex, so a measure combining multiple CVI can also be a single component of the quality vector. To be interesting in a set of clusters, a cluster must not only be \"good\" to some extent (most likely measured by a Quality function) but also unique in some kind of way. If we consider the three clusterings in Figure 4, we can see that two of these three (Figure 4b,c) \begin{table}
\begin{tabular}{c c} \hline \hline
**Notation** & **Definition** \\ \hline \(\mathbb{U}\) & Set of all clusterings \\ \hline \(c\in\mathbb{U}\) & Clustering \\ \hline \(\mathcal{P}(\mathbb{U})\) & Power set of \(\mathbb{U}\) \\ \hline \(C\subseteq\mathbb{U},\,C\in\mathcal{P}(\mathbb{U})\) & Set of clusterings \\ \hline \(Q(c):\mathbb{U}\rightarrow\mathbb{R}^{k},\,k\in\mathbb{N}\) & Quality function \\ \hline \(I(C):\mathcal{P}(\mathbb{U})\rightarrow\mathbb{R}\) & Interestingness function \\ \hline \(S(C):C\rightarrow\mathcal{P}(C)\) & Skyline function \\ \hline \(r(v):\mathbb{R}^{k}\rightarrow\mathbb{R},\,k\in\mathbb{N}\) & Reduction function \\ \hline \(sim(c,o):\mathbb{U}\times\mathbb{U}\rightarrow\mathbb{R}\) & Similarity function \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Used notations. clusterings have a very strong similarity and only differ in the labelling of one object, whilst the other clustering (Figure 4a) is vastly different. If we need to select two clusters from this set, it would grant little to no extra value for a practitioner if these clusters are as shown in Figure 4b,c, since it is highly unlikely that the single changed object will lead to many new insights. The other clustering, however, even while intuitively \"worse\" than the other two, would grant more insight as an addition to the other clustering (furthermore, it might even be the better fitting clustering, if Property B turns out to be irrelevant.).