# Automated Exploratory Clustering to Democratize Clustering Analysis
## 3 Materials and Methods
### AutoML Lifecycle  
The general goal of AutoML can be considered to be solving the _Combined Algorithm Selection and Hyperparameter Optimization_ (CASH) [4] problem, i.e., finding both the perfect algorithm and hyperparameters for a user's problem. This takes the cumbersome activity of fitting a good machine learning model from a data scientist to a computer and hence enables more inexperienced users to find good solutions. **Definition 1** (CASH-Problem). : _Given a clustering \(c\), a Quality function on a solution \(Q(c)\), an algorithm \(A^{i}\in\mathcal{A}\), and \(\Lambda^{i}\), the set of possible hyperparameters for \(A^{i}\), the CASH-Problem can be formulated as follows:_  
\[A^{*}_{\lambda^{*}}=\operatorname*{argmax}_{A^{i}\in\mathcal{A},\lambda\in \Lambda^{i}}Q\Big{(}A^{i}_{\lambda}(D)\Big{)}=\operatorname*{argmax}_{c\in \mathbb{U}}Q(c), \tag{2}\]  
_where \(A^{*}_{\lambda^{*}}\) is the optimal algorithm with the optimal parametrization \(\lambda^{*}\), and \(\mathbb{U}\) is the set of all investigated solutions._  
In practice, many approaches try to solve this problem using an optimization cycle like the one illustrated in Figure 2. When using the framework, a user inputs a dataset and a time budget. The hyperparameter optimizer, Quality function, used algorithms, and their hyperparameter ranges are generally determined by the framework, although some frameworks also enable the user to make decisions on these. The optimizer will select promising combinations of algorithm and hyperparameters and use these to cluster the dataset. The goodness of these clusterings is measured by the Quality function. If the time budget is not exceeded, the optimizer will be updated based on the goodness of the previous clusterings. New algorithm and hyperparameter combinations are then chosen by the optimizer, where the values of the Quality function on the previous combinations are used to improve the quality. When no time is left, the best solution according to the Quality function is returned. In the current Automated Clustering literature, a solution is a single clustering of a dataset, whilst the Quality function is often either a single CVI [19; 20], a consensus, or another combination of multiple CVIs [18]. Obviously, modern Automated Clustering approaches are more sophisticated and employ different techniques such as meta-learning or constraints given by the user, but for this paper, this abstraction will suffice to display our criticism on this model and to define the differences to our AEC problem. Clustering can be used for different tasks [15]. On the one hand, in sophisticated machine learning pipelines, clustering is often used as a preprocessing step and the quality (or usefulness) of a clustering can be evaluated by evaluating the effect of the clustering on the performance of the complete pipeline (if it can be measured). On the other hand, clustering is often used as an exploratory technique to find new insights in the structures of the data. Here, it is not possible to objectively evaluate the usefulness of a clustering, since the same insight might be very useful for one practitioner but trivial or even completely meaningless for another. Some consider that a \"natural\" clustering exists on any dataset [55]. However, this is, even if true, very dependent on the used distance and the notion of similarity, which are both non-trivialchoices even for experienced data scientists. Consider for example the dataset in Figure 3, where every object has three properties as follows: position, shape, and colour. Without any further information, we can see three clear \"natural\" clusterings, where one is based on the position, one on the colour, and one on the shape. These clusterings all can be viewed as \"correct\", but they are three completely different clusterings with not even the same number of clusters. One could even argue that all combinations of these clusters, resulting in 12 different clusters, would be the \"natural\" clustering.